<p>Large-scale multi-modal pretraining model, such as CLIP, has shown remarkable generalization in vision-language tasks. However, the transfer of large models to downstream tasks requires large-scale computing resources, so adapter is proposed to realize fine-tuning for downstream tasks. As input text, prompt can guide CLIP learn the correspondence between images and texts when performing specific tasks. The selection of prompt template is very sensitive. In this paper, we propose Unified Multi-modal Prompt with Adapter For vision-language models (UMPA) based on CLIP, for parameter-efficient fine-tuning (PEFT). Learnable prompt design can improve the adaption ability of model. Adapter can realize lightweight model fine-tuning. With applying learnable prompt and adapter both on vision and language branches while the entire pre-trained parameters freeze, the model improve alignment between the vision and language representations. The model can learn separate visual representations across different stages to extract stage-wise feature relationships progressively, achieve comprehensive learning of rich visual information. Considering the additional learning modules, we apply image attention masking to prevent overfitting in downstream tasks. Experiments and extensive ablation studies on various visual classification tasks demonstrate the effectiveness and strong few-shot generalization ability of our approach.</p>