A Detail-Guided Multi-source Fusion Network for Remote Sensing Object Detection

Optical and synthetic aperture radar (SAR) remote sensing have established themselves as valuable tools for object detection. Optical images exhibit weather-dependence but offer intricate information, whereas SAR images are weather-independent but may exhibit speckle noise and weaker edge details. Combining them can significantly enhance object detection precision. Nonetheless, existing fusion techniques frequently introduce noise of SAR images into fused features, consequently impinging on the efficacy of object detection. In this study, we present an innovative object detection architecture. It extracts detailed richness maps from optical images, subsequently employing these maps to recalibrate the spatial attention weights assigned to optical and SAR features. This strategic adjustment mitigates the impact of SAR noise on fused features within regions abundant in optical intricacies. Moreover, prevailing public optical-SAR fusion datasets need more meticulous instance-level object annotations, rendering them unsuitable for fulfilling object detection. Thus, we introduce two distinct datasets: OPTSAR, characterized by high registration accuracy, and QXS-PART, offering a counterpart with lower registration accuracy to validate the efficiency and generalization of our method. Both datasets encompass instance-level labels for diverse entities such as ships, aircraft, and storage tanks. Empirical assessments conducted on the OPTSAR and QXS-PART datasets underscore the prowess of our method. It substantiates a marked enhancement in object detection precision across well-aligned and poorly aligned optical-SAR fusion scenarios. Our method notably surpasses the efficacy of single-source object detection methodologies and established fusion approaches in terms of accuracy.