A Lightweight Text Classification Model Based on Label Embedding Attentive Mechanism

This paper presents a lightweight model based on the self-attention mechanism for text classification tasks. In our model, we incorporate auxiliary information of the label through the label embedding method, enabling the model to capture the contextual language variations of the same word. Furthermore, we address the issue of misclassification of similar texts by introducing the contrastive loss function, in conjunction with the traditional cross-entropy loss function. Experimental evaluations are conducted on multiple datasets, comparing our model against others with similar parameter scales, thus demonstrating the effectiveness of the proposed approach.