[Proposal] Obstacles & Special Vehicle Detection using V2X

[damodarRohit]

On behalf of Danlaw Inc., I would like to propose implementation of a feature to detect surrounding obstacles using V2X communication and identify the presence of special vehicles (police cars, maintenance vehicles, ambulances, etc.).

An initial prototype version of this implementation has been completed and can be found at Autoware Discussion #4044

Motivation

Current methods of sensing (Lidar, vision-based & radar) have high computational overhead but are unable to reliably detect and classify special vehicles in an emergency mode or not. Integration of V2X capability will enable more robust and reliable detection of these scenarios.
The proposal is to add a V2X Sensing and Perception module to the existing Autoware stack.
The V2X ITS standard has BSM message defined, which can be used for this application. All vehicles equipped with a V2X OBU broadcast BSM as a standard.
Part I of the BSM message contains basic vehicle information like GNSS co-ordinates, vehicle speed & orientation, dimension info., etc. This can be used for reliable object detection and can be fused with other sensor data.
Part II of BSM contains Special Vehicle info and can be used to identify Special Vehicles like Emergency vehicles, Maintenance vehicles, Garbage trucks, etc. This info. can be used for improving the planning capabilities in special situations.

High-level Architecture

V2X OBU Driver

This module will receive decoded V2X packets from the OBU and publishes V2X Messages as required by the V2X Sensing/ Perception modules.

• Input: Decoded V2X Packets from OBU
• Output: /v2x/usa/ [/v2x/]

V2X Sensing Module

This module will process V2X messages and publish data in Autoware specific message format. This module provides abstraction of data between different regions. For the Special Vehicle Detection application, the input & output message types are as follows:

• Input: /v2x/usa
• Output: /sensing/v2x/surround_vehicle_info

V2X Perception Module

The perception module will receive all BSM messages and identify all relevant vehicles and generate a list of objects. The objects will be appended with a category for normal vehicles/ special vehicles based on BSM Part II content.

• Input: /sensing/v2x/surround_vehicle_info
• Output: /perception/v2x/detected_objects

Special Vehicle Planning Module

This module may be part of the planning module and will send specialized commands for planning/ re-routing based on information received from V2X Perception module.

if special_vehicle is category_move_over:
	->publish lane_change
	->publish lane_speed_restrictions
elseIf special_vehicle is category_frequent_stops:
	->trigger_lane_change OR -> trigger_speed_limit
elseIf special_vehicle is category_stop OR category_speed_restricted
	->trigger_speed_limit

Sequence Diagram

sequenceDiagram

participant Emergency Vehicle OBU
box rgb(100,100,100) Autoware Ego Vehicle
participant Ego Vehicle OBU
participant AW V2X Driver
participant AW V2X Perception
participant Special Vehicle Planning
end
Emergency Vehicle OBU--)Ego Vehicle OBU:V2X Broadcast
Note right of Ego Vehicle OBU:Decode V2X Message
Ego Vehicle OBU->>AW V2X Driver:UDP Packet 
Note right of AW V2X Driver:Convert UDP Packets to<br>AW V2X ROS message 
AW V2X Driver->>AW V2X Perception:/sensing/v2x/usa<br>-aw_bsm.msg
Note right of AW V2X Perception:Look-up V2X message and<br>package location & vehicle<br>info to object list 
AW V2X Perception->>Special Vehicle Planning: /perception/v2x/detected_objects
Note over Special Vehicle Planning: Look-up BSM Part II content<br><br>If special vehicle<br><br>Do one or more of below actions<br>-Lane speed restriction<br>-Pull-over<br>-Lane change

Loading

[isamu-takagi]

The discussion of the traffic light may be helpful. In that discussion, for the planning component, it would be preferable if data could be handled in the same way regardless of its source such as V2X, camera, etc. So, the other option is to create a merge node within the perception component to merge the topics.

For special vehicles info, it is possible to represent it by extending the object message such as ObjectClassification.msg.

[mitsudome-r]

@damodarRohit Thanks for the proposal. I have some questions regarding the vehicle labels in ObjectClassification.msgs

Could you explain in more detail about what the following type means?

uint8 FREQUENT_STOPS = 9
uint8 TRAFFIC_STOP_PULLOVER = 10

Also, do you have any reasons why you would like to add a new field as special_vehicle_label rather than defining it as one of the type in label field?

[isamu-takagi]

@damodarRohit
Thank you for updating the proposal. The high-level architecture looks good to me.

For detailed design, one thing to consider is which stage of the three types of objects handled by perception (detected, tracked, predicted) should be supported by V2X. I'm not familiar with use cases of these objects, so does anyone have any opinions?

[damodarRohit]

@mitsudome-r

Could you explain in more detail about what the following type means?

uint8 FREQUENT_STOPS = 9
uint8 TRAFFIC_STOP_PULLOVER = 10

FREQUENT_STOPS refer to vehicles like garbage trucks, snow plows or similar which may stop frequently or have slower speed limit than rest of the traffic. This may be used while Planning to anticipate stops or do a lane change maneuver. This kind of information is currently supported in ITS_NA standard.

TRAFFIC_STOP_PULLOVER may be used by planning module to stop Ego vehicle/ pullover off the road/ lane. This may be used in scenarios like pulling over to the side and stop to make way for emergency vehicles approaching from behind (Ambulance, Police vehicles) or legal stop requests by law enforcement vehicles. This message is more like a future provision and information may be provided by V2X or a different sensor.

Also, do you have any reasons why you would like to add a new field as special_vehicle_label rather than defining it as one of the type in label field?

The information in special_vehicle_label may be different from object class label. Only certain classes of objects (cars, trucks, etc) will have this additional label.The planning module will have to look at label field to identify the class of the object. The special_vehicle_label is used for specific scenario selection like lane change or pullover by the planning module.
All fields related to special vehicles may also be packaged into a SpecialVehicleInfo.msg inside ObjectClassification.msg.
This is a proposal and I would really appreciate any inputs for changes.

[evshary]

@damodarRohit That is a great proposal!
I'm not familiar with the V2X standard, so I have some questions:

1. The proposal mainly focuses on V2V or V2P. Is it possible to include V2I (i.e. traffic light mentioned above)? From my first thought, maybe AW V2X Perception can not only link to AW decorative_tracking_merger but also traffic_light_arbiter. Just wondering whether we can make it more generalized.
2. One of the goals of AW V2X Perception is to handle the region V2X standard. Is there any reason why we don't want to process it in AW V2X Driver?
3. Now we only receive data from OBU to Autoware. Is there any case we need to support the other direction, that is, sending data from Autoware to OBU?

Thank you!

[damodarRohit]

@evshary Thanks for the comment and the valuable feedback.

The proposal mainly focuses on V2V or V2P. Is it possible to include V2I (i.e. traffic light mentioned above)? From my first thought, maybe AW V2X Perception can not only link to AW decorative_tracking_merger but also traffic_light_arbiter. Just wondering whether we can make it more generalized.

Yes, there is a possibility to include V2I or other V2X communication. This proposal is aimed at specific V2V use case and not for complete V2X interface. For instance, the AW V2X perception stack shown above may also handle other perception tasks related to V2X (traffic light information, route information, etc.).

One of the goals of AW V2X Perception is to handle the region V2X standard. Is there any reason why we don't want to process it in AW V2X Driver

Yes, thanks for pointing that out. The initial thought process was to use the AW V2X driver as a hardware interface but I agree with your suggestion. I can update the design based on this feedback.

Now we only receive data from OBU to Autoware. Is there any case we need to support the other direction, that is, sending data from Autoware to OBU?

Yes, there are cases which needs to support messages being shared by Autoware to the V2X stack. For instance, sharing sensor data for co-operative perception, platooning, etc. This again, is not considered under this use-case.