RSA-fusion: radar spatial attention fusion for object detection and classification

Object detection and classification in urban transportation scenarios lack robustness on a single sensor under extreme weather conditions. Radar-vision fusion is a promising routine for its informational complementarity. Previous feature-fusion routines mostly use concatenation, element-wise add, and multiply, which ignores the potential association between multimodal information. In order to solve the problem above, this article proposes a novel method to fuse the features from mmWave Radar and vision sensor. Preprocessed radar features are utilized as gated units for corresponding image features, which realize the focus on important spaces of image features. In addition, a channel attention is added in tandem form to realize the focus on important channels of image features. Experiment results show that the mAP of the advocated fusion algorithm is 1.44% and 1.11% higher than the single-vision algorithm in night and rain scenarios. The proposed algorithm can be robust to extreme weather conditions and be deployed on autonomous driving systems to provide foundations for following route planning tasks.