Cannot call IRGBDSensor::getRgbResolution() when using a gazebo_yarp_camera plugin

[xEnVrE]

I am trying to retrieve the resolution of a camera running in simulation from a robot using the gazebo_yarp_camera plugin.

I am connecting to the camera with a PolyDriver using the IRGBDSensorInterface.

While the calls IRGBDSensor::getRgbWidth() -> int and IRGBDSensor::getRgbHeight() -> int return the correct values, the method IRGBDSensor::getRgbResolution(int&, int&) -> bool always return false.

After a bit of investigation, it seems that the rpc responder responsible for that call is calling getRgbResolution on a yarp::dev::IRgbVisualParams instance whose default implementation is to return false - and that method is not pure virtual, hence it might be that the simulation plugin is not re-implementing that call (but I might be wrong as I am not really an expert on client/server architectures of rgb devices in YARP).

What I noticed is that the gazebo_yarp_camera plugin, more specifically the GazeboYarpCameraDriver class is inheriting from yarp::dev::IFrameGrabberImage. The latter interface has, among others, two methods height() and width().

I could not reconstruct the missing path from IFrameGrabberImage to the IRgbVisualParams unfortunately. Is anyone willing to clarify this connection? Am I using the wrong device interface IRGBDSensorInterface to connect to a simulated camera or is the plugin missing the implementation of some methods / interfaces?

I am available for a PR in case someone could help understand the missing links :)

cc @traversaro

[traversaro]

I am connecting to the camera with a PolyDriver using the IRGBDSensorInterface, i.e. as "nwc" (network wrapper client)?

Which device are you using for opening the PolyDriver? RGBDSensorClient or frameGrabber_nwc_yarp ? Which version of YARP?

[xEnVrE]

I am using RGBDSensorClient just because I was used to do that on the real robot.

Version of YARP is 3.8.1.

[traversaro]

Sorry, last question, which model are you using for simulation? Which gyp version?

[xEnVrE]

I am using ergoCubGazeboV1_1 and for gyp I am using latest devel.

The camera I am interacting with is, I guess, exposed by this wrapper loaded by the yarprobotinterface plugin.

[traversaro]

I am using ergoCubGazeboV1_1 and for gyp I am using latest devel.

The camera I am interacting with is, I guess, exposed by this wrapper loaded by the yarprobotinterface plugin.

Thanks, this is what I was looking for! So the server side is rgbdSensor_nws_yarp, and the complete call is:

Ok, in the case you are using RGBDSensorClient and YARP 3.8.1 the call path should be:

• https://github.com/robotology/yarp/blob/v3.8.1/src/devices/RGBDSensorClient/RGBDSensorClient.cpp#L358 (nwc process)
• https://github.com/robotology/yarp/blob/v3.8.1/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Forwarder.cpp#L59 (nwc process)
• https://github.com/robotology/yarp/blob/v3.8.1/src/devices/RGBDSensor_nws_yarp/RgbdSensor_nws_yarp.cpp#L62 (gazebo process)
• https://github.com/robotology/yarp/blob/65a3a508c0b5147bdf5106ac173370382486004e/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Responder.cpp#L82 (gazebo process)
• https://github.com/robotology/yarp/blob/v3.8.1/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Forwarder.cpp#L74 (nwc process)

Not sure what the false came out, but my guess is that we should implement the proper interface for the gazebo_yarp_camera, as done for the gazebo_yarp_multicamera in #558 .

[traversaro]

Not sure what the false came out

Probably in https://github.com/robotology/yarp/blob/65a3a508c0b5147bdf5106ac173370382486004e/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Responder.cpp#L30 ? Did you checked if gazebo prints that error?

[traversaro]

Or perhaps we should just use gazebo_yarp_multicamera also on ergoCub as that device provides all the functionality of gazebo_yarp_camera? Not an expert of that part, perhaps @Nicogene or @randaz81 may know more.

[xEnVrE]

I checked but the error is not there. Moreover, some of the other calls are actually working, e.g., I am getting the correct answer for

https://github.com/robotology/yarp/blob/65a3a508c0b5147bdf5106ac173370382486004e/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Responder.cpp#L46

and

https://github.com/robotology/yarp/blob/65a3a508c0b5147bdf5106ac173370382486004e/src/devices/framegrabber_protocol/yarp/proto/framegrabber/RgbVisualParams_Responder.cpp#L54

I am also testing them without C++ using yarp rpc directly, basically mimiking the forwarder.

I can get the width, or the height using h, with

yarp rpc /ergocubSim/depthCamera/rpc:i
visr get w

and I get

Response: [visr] [w] [is] 640

However, if I try to get the resolution

yarp rpc /ergocubSim/depthCamera/rpc:i
visr get res

I get

Response: [fail]

[xEnVrE]

Not an expert of that part, perhaps @Nicogene or @randaz81 may know more.

Indeed maybe we can ask them first to understand a bit better! Thanks for the help in understanding the list of calls along the process!

[Nicogene]

Looking at the code it seems that gazebo_yarp_camera does not inherit from IRGBVisualParams or the IRGBDSensorsInterface so it surprise to me that the poly->view() is working.

@randaz81 are IFrameGrabber and IRGBVisualParams sharing the same protocol?
In any case since gazebo_yarp_camera implements only the width and height methods I think it is correct that visr get res fails

[xEnVrE]

IRGBVisualParams or the IRGBDSensorsInterface so it surprise to me that the poly->view() is working.

Isn't that because there is an extra layer made by the rgbdSensor_nws_yarp at the server side? - I mean that the yarprobotinteface plugin is initializing that wrapper that in turn is getting data from the Gazebo device implementing IFrameGrabber.

[Nicogene]

IRGBVisualParams or the IRGBDSensorsInterface so it surprise to me that the poly->view() is working.

Isn't that because there is an extra layer made by the rgbdSensor_nws_yarp at the server side? - I mean that the yarprobotinteface plugin is initializing that wrapper that in turn is getting data from the Gazebo device implementing IFrameGrabber.

Yes, you are right, but then under the nws the device does not implement the getResolution method, this is why it fails.
And probably yes there is one rpc protocol for both interface