Multiple network interfaces on ROS 2

[tokk-nv]

I’m trying to run Rviz2 on a separate x86 Ubuntu 20.04 desktop machine to visualize the Create 3 movements, but I’m facing an issue.
The problem is that the SBC on Create 3 has two separate network interfaces, and messages originating from Create 3 cannot get to the other network on which the x86 machine is.

So if you set up a Raspberry Pi 4 to be the SBC on Create 3, RPi 4 will have two network interfaces; usb0 and wlan0.
RPi establishes the virtual Ethernet connection with Create 3 via usb0 interface.
And we may have the x86 desktop machine on a Wi-Fi router network, where RPi 4 connects via its wlan0.

                Router
                   │
┌───────────────┐  │  ┌────────────────┐       ┌──────────────┐
│  x86 Linux    │  │  │     RPi 4      │       │   Create 3   │
│         ┌─────┤  │  ├─────┐   ┌──────┤       ├─────┐        │
│         │NIC0 ├──┴──┤wlan0│   │ usb0 ├───────┤     │        │
└─────────┴─────┘     └─────┴───┴──────┘       └─────┴────────┘
    192.168.1.11   192.168.1.10 192.168.186.3   192.168.186.2

The issue is when you run ros2 topic list on the x86 machine, it does not find the Create 3 topics.
Do you know how we can solve this situation?

For now, I’m just setting my CYCLONEDDS_URI to be like this.

export CYCLONEDDS_URI='<CycloneDDS><Domain><General><NetworkInterfaceAddress>usb0</NetworkInterfaceAddress></General></Domain></CycloneDDS'

I believe it’s basically this question.
https://answers.ros.org/question/359804/ros2-how-to-forward-route-data-over-network-interfaces-to-communicate-with-multiple-machines/

The short-term milestone of Cyclone DDS lists “Multi-Network interface cards support (Consolidation)”, under Cyclone 0.8.
https://github.com/eclipse-cyclonedds/cyclonedds/blob/master/ROADMAP.md#short-term-milestones
I wonder if this is what is essentially required and is already being offered (hence just the matter of configuring Cyclone DDS right to handle multiple network interface cards?).

[alsora]

HI @tokk-nv, thank you for the great question!

First of all, you need to make sure that all machines (create 3, Raspberry Pi and laptop) are connected to the same Wi-Fi network.

The easiest solution to address this problem is to switch RMW implementation from CycloneDDS to Fast-DDS.
With Fast-DDS multiple networks support works out of the box and there is no need to use XML configurations.

• To change RMW implementation on the Create 3 robot, you should connect to the webserver interface and do that from the "Configuration" tab.
• To change RMW implementation on your laptop or on the RPi4, you should run export RMW_IMPLEMENTATION=rmw_fastrtps_cpp.

On the other hand, if you want to use CycloneDDS, things are a little bit more complicated especially due to the fact that we didn't find any documentation that describes this use-case.

Having said that, the following should work:

• Make sure that on RPi4 you are using CycloneDDS 0.8.0 or higher. This is probably not the default version that ships with ROS 2 Galactic.
• Export the following XML profile using the CYCLONEDDS_URI environment variable:

<CycloneDDS>
   <Domain>
     <General>
        <NetworkInterfaceAddress>usb0,wlan0</NetworkInterfaceAddress>
    </General>
   </Domain>
</CycloneDDS>

Note how we are specifying both network interfaces in the NetworkInterfaceAddress section.
IMPORTANT: the application will crash if any of the specified interfaces is not present.

[dave20874]

@alsora can you explain what you mean by "With Fast-DDS multiple networks support works out of the box?" I have a network setup similar to the original poster and I've configured all the ROS2 elements to use Fast-DDS. But, while the node RPi-4 is able to publish and subscribe to topics on "x86 Linux" and "Create 3", the host "x86 Linux" can't see any topics published by "Create 3". Nor is there direct IP connectivity between those nodes as the link 192.168.186.0 is not reachable from our main network, 192.168.1.0. Is there any sort of DDS configuration that would enable forwarding ROS2 traffic and topics through a network with this topology?

[kengregson]

Same experience with Fast DDS as @dave20874 here with Ubuntu/galactic on both x86 and RPi4.

Have also noticed that with cyclone (galactic's 0.8.0 on x86) configured as described above, subscribing to a create3 topic on RPi (eg. via 'ros2 topic echo ') can cause that topic to also become visible from x86.

Any chance ip_forwarding and/or routes might be required or helpful on RPi (and/or create3/x86)?

[justinIRBT]

@dave20874 @kengregson Did you connect your Create3 wifi to the same network your laptop and RPi-4 is on? The RPi-4 DDS layer won't bridge the communication between the laptop and the Create3, those two need to be on the same network to see each other. If you have a use case where you do not want to connect the Create3 to your wifi for some reason, another option is to make a repeater node on the RPi-4 that subscribes to topics from one system (say Create3) and re-publishes on new topic names, then the laptop should be able to see it.

[kengregson]

@justinIRBT Will give that a try, the create wasn't connected to the WiFi - imagined the RPi might act as a gateway. Any chance the create will bridge DDS topics to/from the RPi on 192.168.186.x? Naive mental model was the create+RPi were a single composite node.

Interesting idea for the repeater node that could be a dynamic ROS2-ROS2 bridge (somehow expected DDS to be that already).

Thanks!

[dave20874]

@justinIRBT Thanks for the clarification. In my case the Create3 was not on the WiFi. (I've also had the same issue with the the Create3 ON the WiFi in a condition where multicasts were blocked between it and my laptop host but that's a separate thing.) I've been able to use topic relays as you suggested. This makes sense now, I just wanted to make sure I wasn't missing anything when I read the OP.