SLAM-VI on MuSHR, looking for advices

[Tomlmgh]

Hello everyone ! We are a team of students working with a company trying to implement a real time SLAM-VI algorithm in a MuSHR robot to try to advance research in this field.
We want to test implementations on the simulator, and then on the robot (we don't often have access to the robot).

So far we have done a state of the art of some SLAM Algorithm we found and we are looking for the help of the MuSHR community to give us your opinions on the algorithm we chose, the feasibility of the implementation on the simu and the robot

Here is our state of the art :

The first 3 are those that seems to be the best to try (for us), ROVIOLI and Vins Fusion seems fine but we don't have time to try many algorithm so we want to focus on the most promising ones, and we exclude the last ones because of the reason listed on the state of the art.

If you have any tips, advice or if you tried something similar we will be happy to welcome them !

Thank you very much for your help.

@Tomlmgh @SashelI @mgodineau @Lilixe @Matoufk
@Arbalestrille

[sidtalia]

Hi, we have found in general is that the MuSHR's current compute capability (Jetson nano) is not particularly suitable for Visual odometry/SLAM. The best FPS we could get for rgbd_odometry with no loop-closure or mapping was 15-20 fps with less than 150 features being tracked. We are currently working on a newer version of MuSHR and we recommend using a fair amount of compute power for real-time VI-SLAM (Jetson AGX Xavier or Orin is recommended, but an i7 processor with good amount of RAM and GPU resources would be ideal if your research allows for it). You can always use the MuSHR for collecting sensor data and running the VI-slam on a desktop if you don't need a real-time system.

That being said, while we haven't tested all of the frameworks listed by you, we have tried using rtabmap-ros package for Visual odometry (which internally implements something similar to ORB-SLAM except that the feature detector is interchangeable with FAST/BRIEF /SIFT/SURF/ORB and so on).
We tried VINS-Fusion (RGB-D I think) but I believe it had some issues with calibration/initialization which have been faced by others as well.
Here's a basic launch file that uses rtabmap's rgb-d VO and here is the corresponding realsense launch file.
I believe rtabmap ros also allows running other state of the art visual odometry frameworks but we haven't really looked into it. Some useful documentation for that has been provided here.

Let me know if you have further questions!
Sidharth Talia.

[SashelI]

Thanks.

Hi, we have found in general is that the MuSHR's current compute capability (Jetson nano) is not particularly suitable for Visual odometry/SLAM. The best FPS we could get for rgbd_odometry with no loop-closure or mapping was 15-20 fps with less than 150 features being tracked. We are currently working on a newer version of MuSHR and we recommend using a fair amount of compute power for real-time VI-SLAM (Jetson AGX Xavier or Orin is recommended, but an i7 processor with good amount of RAM and GPU resources would be ideal if your research allows for it). You can always use the MuSHR for collecting sensor data and running the VI-slam on a desktop if you don't need a real-time system.

That being said, while we haven't tested all of the frameworks listed by you, we have tried using rtabmap-ros package for Visual odometry (which internally implements something similar to ORB-SLAM except that the feature detector is interchangeable with FAST/BRIEF /SIFT/SURF/ORB and so on). We tried VINS-Fusion (RGB-D I think) but I believe it had some issues with calibration/initialization which have been faced by others as well. Here's a basic launch file that uses rtabmap's rgb-d VO and here is the corresponding realsense launch file. I believe rtabmap ros also allows running other state of the art visual odometry frameworks but we haven't really looked into it. Some useful documentation for that has been provided here.

Let me know if you have further questions! Sidharth Talia.

Thanks. We haven't been responding but we considered your answer !
We managed to make orbslam 3 work in both realtime and offline, but only with the cameras (had some issues with the jetson).
As it is a school project, we will make a report on what we did (and how), we'll post it here if someone's interested in it.

[sidtalia]

Would love to see the report when you're done!

[SashelI]

Would love to see the report when you're done!

Hi, the time we had to work on the project came to an end, so as promised here is the final report.
Note that it is in french (as we are french students), and was made for our school, so it may contain some sections you're not interested in.

Regards,
Amélie