FADSiamNet: feature affinity drift siamese network for RGB-T target tracking
摘要
Significant advancements have been announced in RGB-T tracking based on Siamese-based networks. However, several challenges persist in the majority of Siamese-based frameworks, including the inability to extract deep-level features, weak representation of target features, and the failure to fully exploit cross-modal information. To tackle these issues, a feature affinity drift Siamese network for RGB-T target tracking is proposed. First, the single-mode SiameseRPN++ tracking framework is extended to a multi-mode framework. Furthermore, the integration of ResNet50 as a feature extraction network enables the acquisition of deep-level target features. Subsequently, an affinity feature enhancement module (AFEM) is proposed to enhance the representation of target features by leveraging affinity, while simultaneously reducing noise in thermal infrared (TIR) images and reconstructing multimodal images through a cascade fusion strategy. Following that, a feature space drift module (FSDM) is designed to enhance the edge representation of the heat source target by segmenting the feature maps of the infrared modal branches and shifting them in four different directions. Finally, a multimodal feature-guided interactive learning module (MFIM) is put forward to leverage the discriminative information from one modality while guiding the learning process of target appearance features in the other modality. This module enhances the network’s attention towards foreground information by mining the cross-modal information within the feature space and channels. The validity of the proposed network is thoroughly verified through extensive experiments on three RGB-T datasets, with our method achieving a Precision Rate (PR) of 90.5