Optimizing Self-training Sample Selection for Euphemism Detection in Special Scenarios
摘要
Euphemism, which is often used to disguise true intentions or soften the tone of speech, has garnered significant attention for its detection in recent years. Current approaches to euphemism detection struggle with uncommon data, where new and rare vocabularies frequently emerge in the test set, leading to decreased model performance. To address these challenges, we propose a novel framework that combines data augmentation, self-training, and a special sample selection method. We expand the original dataset using lexical and sentence-level data augmentation techniques, enhancing the representation of rare vocabulary and allowing the model to familiarize itself with their contexts and usage characteristics. Next, we employ self-training as a semi-supervised learning approach, utilizing the augmented datasets to iteratively generate and refine pseudo-labels. During each iteration, a KNN framework is used to generate pseudo-labels by searching for the nearest neighbor texts, thus enhancing the quality of these labels. Finally, to further increase the presence of rare vocabulary in the training data, we implement a sample selection strategy based on euphemism frequency and predictive uncertainty, integrating it with the earlier steps. Experimental results demonstrate that our framework outperforms previous research on most metrics and achieves a 10% increase over the baseline model.