We introduce a novel framework for addressing the challenges in 6D pose estimation for pick-and-place manipulation. The proposed vision-based approach leverages a Bayesian combination of SE(3) tracking and SO(3) estimation. This aims to achieve a balance between accuracy and computation time, a common trade-off in existing solutions. Experimental results demonstrate superior performance compared to current state-of-the-art methods, highlighting the proposed framework's effectiveness in high-frequency pose estimation. This is the code base for the project.
The following setup works on Linux only. However, you can modify the CMakeLists.txt yourself to get it running on your OS.
Please install the following in your system
- pcl>=1.3
- eigen>=3.3.9
- openCV
- Boost
cd classical_6d
mkdir build
cmake ..
make -j8
We have provided a requriements file for setting up the python environment.
mkdir venv
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
We are using mustard data. Download the following: YCB Video Data: https://archive.cs.rutgers.edu/archive/a/2020/pracsys/Bowen/iros2020/YCB_Video_data_organized/0050.tar.gz Ply Files: https://archive.cs.rutgers.edu/pracsys/se3_tracknet/YCB_models_with_ply.zip Synthetic Training Data: https://archive.cs.rutgers.edu/archive/a/2020/pracsys/Bowen/iros2020/YCB_traindata/mustard_bottle.tar.gz Weights: Download our weights from Supplmentary Materials.
You are ready to run the code now! Please ensure that the paths are setup correctly for reading the inference datasets.
python3 src\main.py
It is important to note that the classical pose estimation code is not included in the repository due to licensing issues. Instead the following code relies on noisy ground truth to demonstrate effectiveness. You can use PoseCNN or whatever in place of that.
As of now we don't have a bug reporting mechanism setup. Please feel free to email the authors, we are more than happy to help you. Please be aware that we do not guarentee any support for MacOS due to our limited knowledge.
Dhruv Parikh
Special Thanks to the Dr. Wen for open sourcing their code on OpenGL rendering!
Copyright <2023>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.