CLIPCMIC: a zero-shot text classification framework based on cross-modal information completion
摘要
Zero-shot text classification effectively alleviates the challenges of data scarcity and the high cost of manual annotation by eliminating the reliance on labeled training data. However, existing methods still face critical challenges in semantic integrity and cross-modal collaboration: the label mapping process suffers from semantic deviation due to the brevity and abstraction of label text; fixed label text embedding methods rely on manual design and are sensitive to embedding positions; matrix-weighted fusion approaches tend to lose deep semantic information; and automatically generated label descriptions lack precision. To address these issues, this paper proposes a CLIP-based framework for cross-modal information completion (CLIPCMIC) for zero-shot text classification. Specifically, this method designs a novel image-label mapping mechanism and utilizes context-based text generation to construct cross-modal information-completed images. Additionally, a dual-pathway enhancement strategy is introduced during the inference phase to improve text semantic perception. To mitigate the sensitivity to embedding positions, an automatic selection method for annotation placement is proposed. Furthermore, to enhance the precision of automatically generated label descriptions, a two-stage keyword automatic selection mechanism (Selection-CLIPCMIC) is introduced. To validate the effectiveness of the proposed method, this study conducts systematic experiments on seven publicly available benchmark datasets. The results show that CLIPCMIC outperforms the state-of-the-art baseline methods LABCLIP and CLIPMulti (MF) across multiple datasets, with performance gains ranging from 0.5% to 11.7%. In addition, we conduct extensive experiments for further verification.