Realsense D435i gazebo 9 plugin

[ashirsat]

Hi,

I am currently trying to use real sense d435i camera for gazebo9 running on Ubuntu 18.04 with Ros-melodic. I have not been successful in finding a plugin for the d400 series.
Is there an already existing one ?
If not, is it planned to be released soon?
Due to current pandemic situation real world experiments are increasingly proving difficult to perform, and this would really help in keeping the testing and usability of Intel realsense at least in Software in the loop for aerial robots.
@RealSenseCustomerSupport, @doronhi

Thanks,
Aniket

[MartyG-RealSense]

The most recent project I know of involving the RealSense D435 in robotics that can make use of Gazebo is the 'Erwhi Hedgehog' project.

https://gbr1.github.io/erwhi_hedgehog.html

Before that, there was SyrianSpock's Gazebo plugin for ROS and the RealSense R200 camera, which itself was an evolution of an official Intel Gazebo plugin for R200.

https://github.com/SyrianSpock/realsense_gazebo_plugin

[ashirsat]

Thanks for the info. @MartyG-RealSense

I had reviewed that project before posting the question. It does not provide a gazebo plugin for the software in the loop testing for d435i. They just use the rviz visualizer for rendering the map from actual realsense d435 camera.

How different would be the plugin for d435 from the one that is there for r200. Would i be able to build d435i plugin it using the r200 plugin?

Thanks,
Aniket

[MartyG-RealSense]

It looks like a .dae model for the D435 would be needed to substitute in for the realsense.dae model of the R200 in the project.

Intel provide CAD models of the RealSense cameras in STEP model file format here:

https://dev.intelrealsense.com/docs/cad-files

Conceivably, you could then use a STEP to dae convertor to convert the D435 model to dae format. Googling for 'convert step to dae' provides some examples of conversion tools for doing so.

Regarding what other changes would need to be made to the values in the model.sdf file, I'm not clear.

[ashirsat]

@MartyG-RealSense I was more concerned about the actual realsense plugin code it would be different from r200 one because the optics changed and it does not have an imu and also i am not sure how to encode the infrared pattern

[MartyG-RealSense]

The R200 is the original ancestor of the D435i (they are both Stereo cameras), so they are similar in that respect, though of course they have a different field of view size and the D435i supports higher resolutions and frames per second speeds. It is true that lack of simulation for the IMU would be another complicating factor.

The RealSense SDK provides a system called software device to simulate cameras in software without having the physical camera, though it probably is not as ideal as Gazebo in terms of simulating robot projects.

https://github.com/IntelRealSense/librealsense/tree/master/examples/software-device

[ashirsat]

@MartyG-RealSense Can you point me to the R200 gazebo plugin? I can start building my plugin for d435i based off on that. If I have the geometry of the realsense d435i, I can probably add the imu to the plugin for d435i. I would also need the orientation of the imu in the actual camera

[MartyG-RealSense]

The link below is the official Intel Gazebo plugin for the R200 that preceded Syrianspock's.

https://github.com/intel/gazebo-realsense

You can get the camera's full calibration details with the "-c" function of the SDK tool rs-enumerate-devices.

https://github.com/IntelRealSense/librealsense/tree/master/tools/enumerate-devices

[ashirsat]

Hi @MartyG-RealSense . Thanks for the link. I will take a look and reach out in case I have any issues or if the plugin works I will keep you in the loop.

[takanotume24]

Hi, I'm also looking for a way to get the D435 to work on the Gazebo9.
I found a description like this, is it helpful?
pal-robotics/realsense_gazebo_plugin#7

[MartyG-RealSense]

@takanotume24 That link looks very interesting - thanks! I don't have the ability to test it myself right now but I'd love to hear if others have success with it.

[takanotume24]

@MartyG-RealSense Turtlebot3 waffle pi + realsense d435 now works in gazebo. Follow the README of the following repository.
https://github.com/takanotume24/turtlebot3_simulations
The dependencies should be resolved by rosdep, but if you have any errors, can you write them in the issue in my repository?

The model with the changes can be found in this(https://github.com/takanotume24/turtlebot3) repository.
diff -> ROBOTIS-GIT/turtlebot3@master...takanotume24:master

[MartyG-RealSense]

@takanotume24 Thanks! I will create an announcement post on the forum front page that links here so that Gazebo users can try out your repository and feed back. Thanks for your contribution :)

[ashirsat]

@takanotume24 Thanks for linking the repo. @MartyG-RealSense From a cursory look through the code, I think that repo will do the job for the plugin side. Its still missing the realsense d435 sdf and dae file for gazebo rendering and frame represenations. Is this something that @MartyG-RealSense could provide in the realsense ros repo from Intel?

Thanks,
Aniket

[MartyG-RealSense]

Hi @ashirsat I am not involved in development as I am on the support team. @doronhi is the developer of the RealSense ROS wrapper.

[takanotume24]

@ashirsat @MartyG-RealSense I'm sorry, I was failing to resolve the dependency. Can you try again following the turtlebot3_simulations/README.md at with-realsense-d435 · takanotume24/turtlebot3_simulations? (If possible, please clear out catkin_ws/src or create a new workspace and try again).

The D435 model uses a forked version of https://github.com/pal-robotics-forks/realsense.

[ashirsat]

@takanotume24 I don't see the realsense d435 plugin or the urdf/sdf in the linked repo. Am I looking at the wrong branch or web hook?

@doronhi Would we be able to get the d435i sdf file in the realsense ros repo itself?

Thanks,
Aniket

[ashirsat]

@MartyG-RealSense @doronhi Can anyone provide any update how to incorporate the d435 in gazebo. @takanotume24 has not linked the plugin nor the urdf file for the realsense gazebo simulation in the previous message repo.

Thanks,
Aniket

[takanotume24]

@ashirsat Is the simulator not working? What errors do you get in your environment when you follow the README of the with-realsense-d435 branch?
If you're looking for the URDF file for D435, it's at https://github.com/pal-robotics-forks/realsense/tree/kinetic-devel/realsense2_description/urdf, and this file combined with Turtlebot3 is my turtlebot3_simuration repository.

[MartyG-RealSense]

@takanotume24 Thank you for your further help with this discussion!

[ashirsat]

@takanotume24 I see what i missed. I was on the wrong branch. I will checkout the branch that you suggested and get back to you.

Thanks for the help.

Regards,
Aniket

[MartyG-RealSense]

Hi @ashirsat Do you require further assistance with this question please, or is it okay to close this case? Thanks!

[ashirsat]

I haven’t yet checked the suggested solution. But you can close the case. If it does not work I will re-open the issue

On Sun, May 17, 2020 at 06:36 MartyG-RealSense ***@***.***> wrote: Hi @ashirsat <https://github.com/ashirsat> Do you require further assistance with this question please, or is it okay to close this case? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1170 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABO2QJW24FYB5CB4XLS6ZSLRR7R5PANCNFSM4MKA4NLA> .

-- Regards, Aniket

[MartyG-RealSense]

Ok, thank you very much!

[rickstaa]

I will also put this here for future reference. An example of how to use the D435i camera in Gazebo can be found here, might people need it. A detailed explanation video created by @issaiass can be found here.

[MartyG-RealSense]

Thanks very much @rickstaa for posting your resource links :)