Camera/Lidar Fusion Based Detection Architecture

[kaancolak]

In the current design, when merging detected objects inside "camera_lidar_fusion_based_detection.launch.xml", there are some unnecessary modules.

Current architecture;

• 3D detections: Feature removed objects coming from CenterPoint/Apollo etc.
• Short-Range Objects: Split closest ROI-Cluster Fusion objects according to the given "split_range"
• Clustering/Objects*: Shape fitted clustering objects( All-unknown, existence probability = 0 )
• DetectionByTracker/Object: Objects coming from DetectiıonByTracker module

But the problem is here, short-range objects are already derivatives from Clustering/Objects, and we don't need to associate it repetitively with 3D detections.

Possible solution :

1. Add trust distance param to ROI-cluster fusion node, only fuse short-range objects given as a parameter
2. Remove object_range_splitter module from "camera_lidar_fusion_based_detection.launch.xml"
3. Remove one of the object_association_merger modules from "camera_lidar_fusion_based_detection.launch.xml"
4. Convert merging architecture to the given below;

It will speed up over-all detection.

Do you have any comments? @yukkysaito @yukke42 @wep21 @mitsudome-r @xmfcx

[yukkysaito]

Looks good. 👍
The reason for the current design is that range spliter is a node that can be used everywhere, including radar fusion, and I did not want to include the same functionality in roi fusion.
However, I agree with the proposal because it is a trade-off for the computational cost.

[kaancolak]

@yukkysaito thanks for your comment. I will create a PR according to this proposal.

[yukke42]

cc @miursh

[badai-nguyen]

@kaancolak thanks you for your suggestion.
Actually I am also considering to update camera/lidar fusion pipeline and some points the same as yours so I would like to share. cc @yukke42 @miursh
Current architecture in case of using centerpoint for camera_lidar_fusion:

After:

Regarding removing object_range_splitter module from "camera_lidar_fusion_based_detection.launch.xml", I have compared results based on tensorrt_yolox and centerpoint using some internal datasets. The result shows non-significant degradation, even improved in some cases. So I also agree to remove it.

After you remove splitter, the actual fusion range will be extended. Then I suppose skip one of downsamples (at least in far region where pointcloud density is decreased) could improve the accuracy in far-distance range. I am also working on evaluation this point.

[kaancolak]

@badai-nguyen -san, thanks for your comment and detailed explanation.

In your proposal about clustering, giving long-range point clouds directly to euclidean clustering definitely increases the resolution of clustered (unknown) objects and the accuracy of long-range small objects but probably increases computational costs(detailed polygons, more objects, more nodelet), I'm wondering about evaluation results.

[badai-nguyen]

@kaancolak I compared about changes on some internal annotated dataset on different distance range and the result shows increase of precision from distance without downsample pointcloud. Since the current long-range detection accuracy still low, the revision shows double-digit of improvement in some testcases.

I also compared the delay time of topics on rosbag with few vehicle in the long-distance on my local machine, however the delay time of clustering/clusters topics right after modification doesn't see any significant affect. One of the comparison result below for reference.
Current :

After changing

[badai-nguyen]

@kaancolak I created PR #2749 for splitting downsample region suggestion above. Could I ask you to review it?

[kaancolak]

@kaancolak I created PR #2749 for splitting downsample region suggestion above. Could I ask you to review it?

Thanks, I would love to review. :)

[kaancolak]

Relevant all proposals merged, I'm closing this discussion.