Intermittent Pub Problem with iRobot Create 3 Running ROS2 Humble 1.0

[scharbonneau8661]

When executing the following command on the iRobot Create 3 with firmware G.4.3 from a virtual machine (VM) running Ubuntu 20.04 with ROS2 Galactic, it executes perfectly each and every time:

ros2 topic pub --once /alfred/cmd_audio irobot_create_msgs/msg/AudioNoteVector "{append: false, notes: [{frequency: 293.66, max_runtime: {sec: 0,nanosec: 250000000}}, {frequency: 329.63, max_runtime: {sec: 0,nanosec: 250000000}}, {frequency: 261.63, max_runtime: {sec: 0,nanosec: 250000000}}, {frequency: 130.81, max_runtime: {sec: 0,nanosec: 250000000}}, {frequency: 196, max_runtime: {sec: 0,nanosec: 250000000}}]}"

When executing the same command on the iRobot Create 3 with firmware H.1.0 from a VM running Ubuntu 22.04 with ROS2 Humble, it executes one time out of two on average.

When it executes correctly, I get the following output on the terminal of the VM where the command is executed:

Waiting for at least 1 matching subscription(s)...
publisher: beginning loop
publishing #1: irobot_create_msgs.msg.AudioNoteVector(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=0, nanosec=0), frame_id=''), notes=[irobot_create_msgs.msg.AudioNote(frequency=293, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=329, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=261, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=130, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=196, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000))], append=False)

When it does not execute correctly, I get the following output on the terminal of the VM where the command is executed:

publisher: beginning loop
publishing #1: irobot_create_msgs.msg.AudioNoteVector(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=0, nanosec=0), frame_id=''), notes=[irobot_create_msgs.msg.AudioNote(frequency=293, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=329, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=261, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=130, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000)), irobot_create_msgs.msg.AudioNote(frequency=196, max_runtime=builtin_interfaces.msg.Duration(sec=0, nanosec=250000000))], append=False)

In a nutshell, when I get the following output at least once (sometimes it appears multiple times), the robot sings its tune:

Waiting for at least 1 matching subscription(s)...

When I don't get this output, nothing happens.

I would appreciate any assistance you can provide to resolve this problem.

Thank you in advance and best regards,

Serge

[alsora]

Hi @scharbonneau8661,

Which RMW are you using in the Galactic and Humble tests?
Note that the your Ubuntu VM will default to rmw_fastrtps_cpp in humble and to rmw_cyclonedds_cpp in galactic, while on the robot you need to manually change the RMW from the webserver (i.e. flashing a new firmware will not change the RMW for you and will keep the latest used).

Besides giving us this information, you should make sure that robots and laptop are using the same RMW.
Communication between different RMWs is not officially supported by ROS.

Anyhow, I think that the problem is not really due to Create 3, but rather to some instability in the RMWs.
The --once flag can produce flaky results when communicating over WiFi.

To increase the robustness you could try to add these flags to your publish command

--no-daemon --spin-time 10

In the meantime, we can also make the cmd_audio subscription on the robot to be of "reliable" type (it's currently best-effort), which should allow to mitigate the problem (but this will require us to release a new firmware, so it will not be available right away).
Created the related ticket to track this change here #356.

[scharbonneau8661]

Thank you Alberto for the quick response.

In both cases, I'm using FASTRTPS on both the robot and the VM.

--no-daemon does not seem to be a flag that can be used with a ros2 topic pub command.

I'll explore the available flags and see if I can come up with something.

Thank you again for help.

[alsora]

Hi @scharbonneau8661, we just released updated firmwares both for Galactic (G.4.4) and Humble (H.1.1) which include a change to the cmd_audio subscription that is located on the robot.

Now the subscription uses a reliable QoS.
This means that you should not have the problem anymore when sending single messages there from the command line.

Note that this is a breaking change: a best-effort publisher won't be able to communicate anymore with this subscription.
The publisher that is used under the hood when you do ros2 topic pub is of reliable type, so this doesn't apply there.

[scharbonneau8661]

Hi @alsora. I just installed the new Humble H.1.1 firmware on my iRobot Create 3 and re-executed the test above. Unfortunately, I get the same intermittent behavior as before. I also tested the Galactic G.4.4 firmware and it works perfectly.

[scharbonneau8661]

I tried to use different flags but I could not find a combination of flags that makes the command work each and every time. I will roll back to Galactic on both the robot and the VM, and I'll wait for an updated version of the Humble firmware for the robot.

Thank you,

Serge

[RealManRobot]

I also have similar problems. When I use node publish one topic, sometimes subscription can't receive. When I pub this topic in terminal, sometimes it will show (Waiting for at least 1 matching subscription(s)...) . Can you tell me a way to solve this problem, Thank you ,

Kaola

[slowrunner]

Suggest you open a new question with your answers to the configuration. It will help you solve the problem faster.

[RealManRobot]

OK，I have been open a new question, and add my code link. Thank you.