Radar-Visual Fusion Network Based on Channel Attention

As research on perception deepens, the challenges of perception under poor line-of-sight conditions are increasing. It is difficult to complete tasks solely relying on visual perception under such conditions, necessitating the use of additional sensors. Millimeter-wave radar possesses all-weather working capability, and the fusion of millimeter-wave radar and vision effectively addresses perception challenges under harsh line-of-sight conditions. This study proposes a method for radar and vision fusion combined with a channel attention mechanism. Firstly, features are extracted separately for vision and radar. Then, when combining radar and video feature maps, a channel attention mechanism is introduced to enable the network to focus more on informative feature channels, thereby further improving recognition accuracy. To meet task requirements, we modified the existing dataset, named Camera-Radar of the University of Washington (CRUW), and evaluated the effectiveness of our proposed method on this dataset, demonstrating its superiority over the existing network structure.