An efficient multi-modal perception based metal fragment detection algorithm in outdoor environments

Current multi-modal based target detection methods for metal debris in outdoor environments easily suffer from specific modal noise interferences and insufficient multiple feature fusion, which result in low recognition effectiveness. To address these challenges, we propose a novel algorithm for metal fragment detection based on visible light and thermal infrared images. Specifically, we firstly design a Bidirectional Frequency-Domain Adaptive Attention Gating (BAFA-Gate) module, to capture the different channel feature correlation and corresponding feature weight vectors among varying modalities, which can be beneficial to reducing modal noise interferences and further improving detection accuracy. In addition, we construct a feasible Global Feature Fusion module (GFF), to perform both intra-modal and inter-modal fusion of normalized features, which can help to obtain global contextual information and enhance the feature fusion capability. Finally, a series of experiments are conducted using the constructed dataset, and the corresponding experimental results validate the effectiveness and feasibility of the proposed method.