PMFNet: collaborative prompt enhancement and dynamic fusion for robust RGB-thermal tracking

Multimodal object tracking, which synergistically utilizes visible-light (RGB) and thermal infrared (TIR) modalities, provides a technical foundation for reliable tracking in complex environments. However, fully exploiting the complementary advantages of these two modalities and achieving effective cross-modal fusion remain challenging, especially under conditions such as occlusion, low illumination, and thermal noise. To address these issues, we propose PMFNet, a two-stage framework for RGB-T tracking that combines modality-aware prompt enhancement with dynamic cross-modal fusion. First, we design two prompt-based modality-specific enhancement modules: the Dual-branch Multi-scale Frequency Prompt Enhancement (DMFPE) module for RGB feature enhancement, and the Thermal Saliency-guided Frequency-separated Prompt Enhancement (TSFPE) module for TIR feature enhancement. Specifically, DMFPE enhances discriminative texture details through grouped convolutions in the frequency domain, while TSFPE refines spatial contours via thermal saliency detection. The learnable prompts then inject this modality-specific information into the Vision Transformer (ViT) backbone. Second, we propose a multimodal multistage weighted fusion (MMWF) module, which decomposes multimodal features into pixel-level semantic, spatial structural, and multi-scale contextual stages, and performs adaptive weighted fusion to dynamically maximize the contribution of the dominant modality. Experiments on the LasHeR, RGBT234, and RGBT210 datasets demonstrate that our method achieves competitive performance. Compared with the strong baseline TBSI, PMFNet improves precision by 1.6% and success rate by 1.3% on average. Overall, this work presents a robust dual-path collaborative optimization strategy that significantly enhances the tracking robustness of RGB-T models in diverse and challenging scenarios. Our code is available at https://github.com/limomo-xuehua/PMFNet and permanently archived on Zenodo with DOI https://doi.org/10.5281/zenodo.18476426.