[Proposal] LiDAR-Landmark localizer using reflectors

[YamatoAndo]

The TIER IV Localization/Mapping team is developing a method for estimating positions using reflectors coated with paint that exhibits strong reflection properties for lasers.
While we already have a Landmark position estimation method using AR tags, I would like to introduce the development of a new Landmark position estimation method that we are currently working on.

Background/Overview

In environments like tunnels, where there are few effective features for LiDAR Scan Matching, there have been many cases where position estimation fails. We aim to achieve stable position estimation even in such environments, and we have decided to install landmarks as a method that allows for easy position estimation. For landmarks, we are considering installing reflectors on the wall that are easily detected based on the intensity of LiDAR pointcloud. The idea is to compare the detected reflector positions with the positions of landmarks specified in the lanelet2 map, allowing for position correction.

Tunnel with reflectors installed on the wall	Visualizing the intensity of LiDAR pointcloud

Detection Method

We are considering detecting reflectors like the ones shown in the image below. These reflectors have highly reflective paint applied to the central vertical line compared to the ends. In our proposed method, we plan to utilize the difference in the intensity to detect the location of these reflectors.

Additionally, due to restrictions under Japan's road law, we use reflectors coated with a special paint (co-operated with Taisei Corporation, Yuri Shimizu, et al.) that is an improved version of the paint complied with the inner walls of tunnels requirements, where high reflection is not visually noticeable. Therefore, it is not something readily available in ordinary circumstances. The special paint is intended for use in tunnels and fulfills various criterions (e.g. non-combustible materials) of Japanese expressway company (NEXCO), and is the only paint that has been used on public roads. When applied to actual tunnels, it is necessary to fulfill the various required criterions of each country. Please refer to these criterions when selecting materials. But if you just want to test this method on a test course, I believe commercially available reflective tape can be used as a substitute.

Reflector

• The tunnel in the above images is National Institute for Land and Infrastructure Management, Full-scale tunnel experiment facility.

Diagram

The proposed system configuration is similar to the configuration for the ARtag-Landmark localizer , and we aim to implement it with high compatibility with the implementation of the ARtag-Landmark localizer.

Lanelet2 Format

The format in lanelet2 is the same as in the case of AR tags. The only difference from AR tags is that we distinguish it by specifying <tag k="subtype" v="reflector"/>.

For the definition of landmarks, it has been proposed in Discussing Landmark Definitions and Code Location for Landmark Based Localizer · autowarefoundation · Discussion #3945 , so please refer to that.

...

  <node id="1" lat="35.8xxxxx" lon="139.6xxxxx">
    <tag k="mgrs_code" v="99XXX000000"/>
    <tag k="local_x" v="22.2356"/>
    <tag k="local_y" v="87.4506"/>
    <tag k="ele" v="2.1725"/>
  </node>
  <node id="2" lat="35.8xxxxx" lon="139.6xxxxx">
    <tag k="mgrs_code" v="99XXX000000"/>
    <tag k="local_x" v="22.639"/>
    <tag k="local_y" v="87.5886"/>
    <tag k="ele" v="2.5947"/>
  </node>
  <node id="3" lat="35.8xxxxx" lon="139.6xxxxx">
    <tag k="mgrs_code" v="99XXX000000"/>
    <tag k="local_x" v="22.2331"/>
    <tag k="local_y" v="87.4713"/>
    <tag k="ele" v="3.0208"/>
  </node>
  <node id="4" lat="35.8xxxxx" lon="139.6xxxxx">
    <tag k="mgrs_code" v="99XXX000000"/>
    <tag k="local_x" v="21.8298"/>
    <tag k="local_y" v="87.3332"/>
    <tag k="ele" v="2.5985"/>
  </node>

...

  <way id="5">
    <nd ref="1"/>
    <nd ref="2"/>
    <nd ref="3"/>
    <nd ref="4"/>
    <tag k="type" v="pose_marker"/>
    <tag k="subtype" v="reflector"/>
    <tag k="area" v="yes"/>
    <tag k="marker_id" v="0"/>
  </way>

...

Autoware Integration Plan

We plan to submit a Pull Request by early March.
We have already created the Pull Request in draft status, but there are still many code modifications to be made.

Thank you for your support, and I look forward to your feedback!

---

This feature is being developed in collaboration with Yuri Shimizu

[xmfcx]

I think modifying existing tunnels to be able to drive autonomously is like cutting bushes near the roads because the ground segmentation algorithm is not performing well.

Instead of focusing the efforts on modifying the existing road infrastructure, we should focus on increasing the performance of real time road feature detection capabilities of the Autoware and be able to drive within an existing lane.

Here a Waymo vehicle is driving in an unmodified tunnel without a driver:

• https://www.youtube.com/watch?v=LYfNi_wYMIM

[YamatoAndo]

@xmfcx
Your perspective makes sense.
It's ideal to enable autonomous driving in existing tunnels without modifications, and I believe it's crucial to advance that development.
However, such development requires a significant amount of time.
On the other hand, there is a high demand for driving through tunnels, and there are requests for it to be realized as soon as possible.
So we are considering modifying the tunnels as a means to achieve this reliably and easily.

[YamatoAndo]

I've created a PR, so please review and leave your comments.
Thank you!
autowarefoundation/autoware.universe#5573

[YamatoAndo]

merged!