DevOps Dojo: ROS Node Configuration - Final3 Proposal

[kaspermeck-arm]

The Open AD Kit WG is working on improving the process of how to configure the ROS nodes. Gathered from discussions and meetings the following Final3 Proposal has been made.

DevOps Dojo: ROS Node Configuration - Final3 Proposal

Background

DevOps: Ros Node Configuration addresses the topic of how ROS nodes are configured. Guidelines, documentation and changes to the parameter files and ROS nodes will be made, based on best-practices from cloud-native and software-defined development methodologies.

How to configure ROS nodes is non-differentiating, and creating alignment in the AWF community will allow developers and users of Autoware to become more productive as less time will be spent on trivial tasks.

flowchart TD
    NodeSchema[Schema file: *.schema.json]
    ParameterFile[Parameter file: *.param.yaml]
    WebDocumentation[Web documentation table]

    NodeSchema -->|Validation| ParameterFile
    NodeSchema -->|Generate| WebDocumentation

Loading

Schema

Each ROS package/node has a schema which describes the node's parameters. The schema will:

• validate the *.param.yaml and itself (i.e., *.schema.json) files as part of the CI
  • the config/ folder can container more than one *.param.yaml file and all must pass the schema's requirements
• be rendered into a table to be used in the web documentation, similar to NDT Scan Matcher

There needs to be a 1-to-1 match between the schema and the parameters. All parameters listed in the schema must be declared in the ROS node, and no parameters may be listed in the parameter file which aren't declared in the ROS node.

Naming convention and file path

• schema file path: autoware.universe/<path_to_package>/schema/<node_name>.schema.yaml
  • <path_to_package> is, e.g., perception/lidar_apollo_segmentation_tvm_nodes
  • <node_name> is the name of the node, e.g., lidar_apollo_segmentation_tvm_nodes

Attributes

The JSON Schema validates the parameters in the parameter file. Some attributes are required and some are optional. The optional attributes are encouraged when applicable.

Required

• name
• type
• description

Optional

• default
• bounds
  • e.g., exclusiveMinimum

Layout

We will use lidar_apollo_segmentation_tvm_nodes as an example and as a base for our modifications. Below is the schema for the node, however not complete, it only has one parameter range.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Parameters for Lidar Apollo Segmentation TVM Nodes",
  "type": "object",
  "definitions": {
    "lidar_apollo_segmentation_tvm_nodes": {
      "type": "object",
      "properties": {
        "range": {
          "type": "number",
          "default": 90,
          "exclusiveMinimum": 0,
          "description": "The range of the 2D grid with respect to the origin."
        }
      },
      "required": [
        "range"
      ]
    }
  },
  "properties": {
    "/**": {
      "type": "object",
      "properties": {
        "ros__parameters": {
          "$ref": "#/definitions/lidar_apollo_segmentation_tvm_nodes"
        }
      },
      "required": ["ros__parameters"]
    }
  },
  "required": ["/**"]
}

Parameter file

The schema will be used to validate the parameter file. The layout below will be used, which is very similar to the layout which is currently used, e.g., see Lidar Apollo Segmentation TVM Nodes parameter file.

/**:
  ros__parameters:
    range: 90

Please note that the parameter file is validated by the schema and it contains all parameters' name and its corresponding default value as required by the schema.

Naming convention and file path

• parameter file path: autoware.universe/<path_to_package>/config/*.param.yaml
  • "*" is the name of the node
• autoware.universe/localization/ndt_scan_matcher/config/ndt_scan_matcher.param.yaml is using the correct naming convention and file path
  • the feature/function is localization
  • the package is ndt_scan_matcher
  • the name of the node is ndt_scan_matcher

ROS node declare parameter function

The default_value in declare_parameter(...) should not be used. If no default_value is provided, the function throws an exception, which is desirable as it enforces the parameter file to contain the declared parameter.

We'll be using lidar_apollo_segmentation_tvm_node.cpp #L43 as the example to show the required change, which can be used in the code with the following pattern:

declare_parameter<int64_t>("range");

For clarity, it is important to stick to using only one of those predefined types in the template, see the table in the section below. Although using a different type compiles just fine (for example, rclcpp correctly infers that int32_t should map to PARAMETER_INTEGER, but the returned value is still an int64_t), it is misleading and could lead developers to make assumptions that result in unexpected runtime behaviors.

Parameter types

The parameter types handled by declare_parameter origin from the definitions in ParameterValue.msg. Those map as:

ParameterType enum	C++ type
PARAMETER_BOOL	bool
PARAMETER_INTEGER	int64_t
PARAMETER_DOUBLE	double
PARAMETER_STRING	std::string
PARAMETER_BYTE_ARRAY	std::vector<uint8_t>
PARAMETER_BOOL_ARRAY	std::vector<bool>
PARAMETER_INTEGER_ARRAY	std::vector<int64_t>
PARAMETER_DOUBLE_ARRAY	std::vector<double>
PARAMETER_STRING_ARRAY	std::vector<std::string>

Example of schema in VSC

I created a short video with a practical example in VSC using the following extensions:

• JSON
• RedHat YAML

This allows a developer to easily create a new parameter configuration file. Find example video here.

Edits

Edit:

• changed PARAMETER_BYTE_ARRAY from unsigned char to uint8_t

Edit2:

• schema to validate instead of generate the *.param.yaml file
• mentions of launch removed as this is outside of the scope of config, this should be addressed in DevOps Dojo: ROS Launch Configuration

Edit3:

• replaced lidar_apollo_segmentation_tvm_nodes with /**

Edit4:

• added validation of *.schema.yaml using JSON Schema in CI

Edit5:

• added required and optional for parameter attributes

[VRichardJP]

• Just to be pedantic, I think the PARAMETER_BYTE_ARRAY should use uint8_t, not unsigned char.

generate the .param.yaml file during build

Does it mean the param.yaml files will be removed from individual packages?

the generated .param.yaml file will be copied to the Autoware Launch repository

I thought autoware_launch is supposed to override the parameters. What is the purpose of the copy? validate the params in autoware_launch?

[VRichardJP]

If the param files are removed from the packages, I am afraid it makes tinkering/tuning more difficult.

At the moment, If I want to test and tune a package, I can modify the package params.yaml located in the src/ directory. To be precise, the param file that matters when a package node is launched is the one in install/ directory, but when --symlink-install is used to compile autoware (and this is the recommended way), the param file in install/ points to the original one in src/. Whatever I do to the install/ directory (clean, rebuild the package), any temporary change I have made to the params.yaml is kept (and tracked by git).
If the params file are only generated at build time, it has 2 consequences:

• If the param files are modified in install/ dir, changes are not tracked and temporary tuning would be erased if the package/project is rebuilt.
• If the param files are modified from the schema side, it is necessary to recompile the package.

I also checked the video. Is there any way to add the schema rules automatically in the .vscode directory? or a magic one line rule that would on all packages at the same time?

[kaspermeck-arm]

@VRichardJP, we definitely don't want to make it more difficult for the developer to tinker/tune. I'll add

• param.yaml file discussion

to the agenda for Thursday's meeting.

If the param files are removed from the packages, I am afraid it makes tinkering/tuning more difficult.

At the moment, If I want to test and tune a package, I can modify the package params.yaml located in the src/ directory. To be precise, the param file that matters when a package node is launched is the one in install/ directory, but when --symlink-install is used to compile autoware (and this is the recommended way), the param file in install/ points to the original one in src/. Whatever I do to the install/ directory (clean, rebuild the package), any temporary change I have made to the params.yaml is kept (and tracked by git). If the params file are only generated at build time, it has 2 consequences:

* If the param files are modified in `install/` dir, changes are not tracked and temporary tuning would be erased if the package/project is rebuilt.

* If the param files are modified from the schema side, it is necessary to recompile the package.

Does the param.yaml file have to live in src/ for it to exist in install/ after compilation?

I also checked the video. Is there any way to add the schema rules automatically in the .vscode directory? or a magic one line rule that would on all packages at the same time?

I wanted to ensure that it was possible to use the tooling in an IDE and I assume that there is a smarter automated way to do this. The work related to this topic would be done in

• DevOps Dojo: Developer (Phase 1) autoware.universe#2653

[VRichardJP]

Does the param.yaml file have to live in src/ for it to exist in install/ after compilation?

It depends if Autoware is compiled with --symlink-install or not. For developers this is a very convenient option, as any change to the parameter or launch files in the package source code takes effect immediately (no need to rebuild anything).

Let's take ekf_localizer as example. The package contains config/ekf_localizer.param.yaml and launch/ekf_localizer.launch.xml which are installed in the package share directory. As I use the symlink install option, this gives this on my machine:

$ ll install/ekf_localizer/share/ekf_localizer/config/
total 4.0K
lrwxrwxrwx 1 vrichard vrichard 108 Mar 27 18:09 ekf_localizer.param.yaml -> /home/sig/autoware/src/universe/autoware.universe/localization/ekf_localizer/config/ekf_localizer.param.yaml
$ ll install/ekf_localizer/share/ekf_localizer/launch
total 4.0K
lrwxrwxrwx 1 vrichard vrichard 108 Mar 27 18:09 ekf_localizer.launch.xml -> /home/sig/autoware/src/universe/autoware.universe/localization/ekf_localizer/launch/ekf_localizer.launch.xml

When I launch the ekf_localizer node (e.g. ros2 launch ekf_localizer ekf_localizer.launch.xml), ROS2 actually uses the launch and params files from the package source. If I didn't use symlink install options, the file would have been copied instead.

This "trick" is widely used in ROS2 projects, it is not just a thing of Autoware.

I don't see how we could keep this behavior if the yaml parameters are generated at build time from the schema. In other words, I think the yaml params should still exist in the package source code. To go even further, if we keep the yaml file instead of generating it with the schema, then I am not sure it makes sense to keep the "default" field in the schema (since the param file would be the "default").

[kaspermeck-arm]

@VRichardJP

Thanks, that answers my question. I see you have the *.param.yaml file in the config/ folder, so the path of the *.param.yaml file in the current proposal doesn't need to change.

The purpose of having default attribute in the schema would be to provide the developer with a recommended/tested value. It is also useful as it auto-populates values for all the parameters which is quite convenient (as shown in the video with the parameter range).

[kaspermeck-arm]

I removed the topic of launch in the proposal as it is a discussion better had in DevOps Dojo: ROS Launch Configuration.

[kaspermeck-arm]

To discuss in meeting March 30th.

I forgot to take Fatih's reason why not to define the node name in the files into account, see https://github.com/orgs/autowarefoundation/discussions/3325#discussioncomment-5406333. This is a good motivation why we would keep the /** and not replace it with, e.g., lidar_apollo_segmentation_tvm_nodes.

[kaspermeck-arm]

Notes from Open AD Kit meeting on March 30th

The AWF community has one week to object to the Final3 Proposal. If there are no objections, then the proposal is officially approved and execution phase can begin. This will be discussed in the March 6th meeting.

Tasks to be done in other dojos:

• DevOps Dojo: CI
  • *.schema.json validation which ensures that the JSON Schema format is valid
  • *.param.yaml validation using the format and requirements outlined in *.schema.json
• DevOps Dojo: Developer
  • add a .vscode folder with a setting.json file associating the with the *.param.yaml and *.schema.json

[kaspermeck-arm]

Additional tasks:

• Documentation
  • JSON Schema to md-table for documentation
  • Contributing guidelines
    • https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/ros-nodes/parameters/

[kaspermeck-arm]

Final3 Proposal has been approved (Open AD Kit meeting on April 6th)!