Status Update: 2023-02-01

[haixuanTao]

🥇

[haixuanTao]

ROS 2 Benchmark retrospective

ROS 2 Shared memory

Shared Memory and Zero Copy:

• Available for c++. So called (rclcpp: ROS Client Library for C++)
• No support for Python. See: Add support for loaned message API to rclpy ros2/rclpy#889
  • Additional problem with python in byte array type: byte[] AKA IDL sequence<octet> fields required to be sequence of bytes instances instead of bytes instance ros2/rosidl_python#134
• Very limited support for Rust. Does not support variable size message such as binary. https://github.com/ros2-rust/ros2_rust/blob/087527ef12a353c5b4023b242c31c715f5369a14/rclrs/src/publisher.rs#L168-L197

it is not possible within ROS2 to switch from shared memory to normal memory as we do with iceoryx and Zenoh. This is linked to the ROS2 API design. You are required to explicitely pass a ros2 shared memory message type and use the borrow_loaned_message API. This makes it necessary to write one code for zero copy node and one without it. See exemple in ros2_rust examples.

Quantitative results

All benchmark I've made on ROS2/CycloneDDS on both Rust and Python shows that it does not use shared memory when not loaning messages. Speed is TCP speed.

Issues with ROS Messages. Bridging generic data type.

ROS message type requires to create bridges to and from generic data type due to its memory layout. It is often not clear if those bridges are zero copy or not.

For exemple, for computer vision bridge, the bridge between OpenCV image type and ROS message. Reading from ROS is zero-copy but creating a ROS message from OpenCV requires a copy.

The main challenge is that ROS message requires ownership of the data. This often means needing a copy.

Apache Arrow to the rescue

Apache Arrow solves a big part of those performance issues by being in the right memory layout, making it zero copy for basic type. See: to_numpy ()

Apache arrow has it's own limitation as well as the data typing might be less verbose. Image information might not include encoding for example.

Apache Arrow zero-copy for Python API

Using arrow for python without copy is being implemented in jorgecarleitao/arrow2#1369 of arrow2. This is unsafe and we will have to double check on safety but using zero copy gives a significant performance advantage compared to other framework.

Implementation details. See: main...slice-arrow

Arrow zero copy for C/C++

Moving our C/C++ API from our own ffi pointer to arrow array and schema pointer should be pretty straight-forward. This removes the need for a copy and potentially removes the need for serialisation (autocore currently use cereal).

Ex: https://github.com/apache/arrow-nanoarrow

Implementation details:

fn to_c_ptr(array: Box<dyn Array>, py: Python) ... {    
    let schema = Box::new(ffi::export_field_to_c(&Field::new(
        "",
        array.data_type().clone(),
        true,
    )));
    let array = Box::new(ffi::export_array_to_c(array));

    let schema_ptr: *const arrow2::ffi::ArrowSchema = &*schema;
    let array_ptr: *const arrow2::ffi::ArrowArray = &*array;
    
    // ...
// Ex from nanoarrow
int print_simple_array(struct ArrowArray* array, struct ArrowSchema* schema) {
  struct ArrowError error;
  struct ArrowArrayView array_view;
  NANOARROW_RETURN_NOT_OK(ArrowArrayViewInitFromSchema(&array_view, schema, &error));

The Array trait

The Array trait is implemented for a large range of datatype, and is transmute-free.

    BinaryArray
    BooleanArray
    DictionaryArray
    FixedSizeBinaryArray
    FixedSizeListArray
    ListArray
    MapArray
    NullArray
    PrimitiveArray (u8, i8, f16 ...,)
    StructArray
    UnionArray
    Utf8Array

Arrow zero-copy for Rust

We can have a relatively close to the other API implementation where data is a <dyn Array> that can be downcasted within the userspace code for input and within our send_output function for output.

let array_trait: Box<dyn Array> 
let data = array_trait
    .as_any()
    .downcast_ref::<PrimitiveArray<u8>>()
    .wrap_err("Could not cast sent output to arrow uint8 array")
    .unwrap()

Notes on dataflow typing.

This means that typing is going to be managed imperatively and not declaratively meaning that it's not going to be present within the dataflow. This has the advantage to make implementation celerity faster. But the inconvenient is that it is not "checkable" with cli by default. We can potentially create macros to generate metadata that can be used to check.

Quick recap of Dora - Arrow vs ROS2

Feature	ROS2	Dora - Arrow
Shared Memory API	ROS2 shared memory API is different from the base API meaning that you cannot use shared memory by just switching the middleware	We can use shared memory by switching the middleware with the same API
Shared Memory Polyglotism	Shared memory is fully supported in c++ but limited in Rust and non existent in Python	We're aiming for zero copy - shared memory on Rust, Python, C/C++
Message type	Requires ownership which mean that it needs a copy from a src pointer	Arrow can create a view of another pointer without copy
Mesage conversions (bridges)	There's no prebuilt conversions from ROS messages to other datatype format and therefore requires custom bridges like cv_bridges	Converting arrow data into other datatype format (numpy, pandas) is really easy as the memory layout is really common and most of the time zero-copy. It can even be free with library that use arrow such as polars...
Computation		Well formatted for SIMD operation and have features for passing data into GPU Buffer

[phil-opp]

dora-daemon

• Extract shared memory server/client implementation into separate crate
• Benchmark latency shared memory server/client implementation
  • Average roundtrip latency: ~3.5µs
  • ➜ shared memory communication is not the bottleneck
• Add special handling for zero-sized messages to avoid one roundtrip
• Limit queue length to 10 events per subscriber
  • Avoids that a slow receiver leads to memory buildup
  • The queue length will be configurable in the future
  • Only drop input events, not other event types (e.g. we don't want to drop a stop event)
• Start porting dora-runtime to dora-daemon
  • Goal: Support existing operator implementations for easier migration. Example: dora-drives
  • Remove special-casing of operator inputs/outputs ➜ make dora-runtime more similar to "normal" nodes