Predictive Score-Guided Mixup for Medical Text Classification

Text classification aims to classify text into one or more predefined categories based on its characteristics. Although existing methods improve model performance by fine-tuning pre-trained language models and introducing label embeddings, due to the sensitivity of medical data, its scarcity often leads to overfitting, as the model tends to overly focus on scarce samples. Details and noise, and cannot generalize well to new data, thus affecting the robustness of the model. To address this issue, we propose a novel approach that integrates prefix label embedding with pretrained language models. Furthermore, we introduce a scoring mechanism for assessing the similarity between labels and text at the classification level. By leveraging predictive score-guided Mixup, our method effectively mines features closely related to classification, alleviating overfitting and enhancing model robustness. Additionally, incorporating multi-head mechanisms enriches feature representation and improves model interpretability. Experimental results demonstrate that our framework significantly improves accuracy on medical datasets.