MPLR: a long-tailed recognition method based on visual language prompts
摘要
Long-tailed recognition is a very challenging research problem with a wide range of applications in the real world. Although CLIP-based methods have achieved state-of-the-art recognition accuracy, they are often quite complex and incur high computational costs. This raises a key question: how can we reduce complexity while maintaining competitive recognition accuracy for long-tailed recognition? To address this problem, we propose a new framework, Multi-Prompt Long-Tailed Recognition (MPLR), which utilizes a pre-trained vision-language model for long-tailed recognition. MPLR consists of two components: Multimodal Prompt Tuning (MPT) and Dynamic Visual Prompt Generator (DVPG). MPT fine-tunes both the visual and textual branches of a pre-trained model by introducing visual and textual prompts. DVPG focuses on identifying tail classes with limited samples by integrating image prompt features and task-related textual prompt features obtained from MPT. Compared with some state-of-the-art methods, our framework has two distinctive characteristics: (1) The parameter size of the model is small and the efficiency is very high. (2) The model has very strong competitive recognition accuracy. The extensive experiments demonstrate that MPLR significantly enhances tail class accuracy with improved training efficiency. Our code is available at https://github.com/SuperLuckyQiu/MPLR.