Euphemism, which is often used to disguise true intentions or soften the tone of speech, has garnered significant attention for its detection in recent years. Current approaches to euphemism detection struggle with uncommon data, where new and rare vocabularies frequently emerge in the test set, leading to decreased model performance. To address these challenges, we propose a novel framework that combines data augmentation, self-training, and a special sample selection method. We expand the original dataset using lexical and sentence-level data augmentation techniques, enhancing the representation of rare vocabulary and allowing the model to familiarize itself with their contexts and usage characteristics. Next, we employ self-training as a semi-supervised learning approach, utilizing the augmented datasets to iteratively generate and refine pseudo-labels. During each iteration, a KNN framework is used to generate pseudo-labels by searching for the nearest neighbor texts, thus enhancing the quality of these labels. Finally, to further increase the presence of rare vocabulary in the training data, we implement a sample selection strategy based on euphemism frequency and predictive uncertainty, integrating it with the earlier steps. Experimental results demonstrate that our framework outperforms previous research on most metrics and achieves a 10% increase over the baseline model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Optimizing Self-training Sample Selection for Euphemism Detection in Special Scenarios

  • Zhiwei Zhang,
  • Shuyu Chang,
  • Haiping Huang,
  • Rui Wang

摘要

Euphemism, which is often used to disguise true intentions or soften the tone of speech, has garnered significant attention for its detection in recent years. Current approaches to euphemism detection struggle with uncommon data, where new and rare vocabularies frequently emerge in the test set, leading to decreased model performance. To address these challenges, we propose a novel framework that combines data augmentation, self-training, and a special sample selection method. We expand the original dataset using lexical and sentence-level data augmentation techniques, enhancing the representation of rare vocabulary and allowing the model to familiarize itself with their contexts and usage characteristics. Next, we employ self-training as a semi-supervised learning approach, utilizing the augmented datasets to iteratively generate and refine pseudo-labels. During each iteration, a KNN framework is used to generate pseudo-labels by searching for the nearest neighbor texts, thus enhancing the quality of these labels. Finally, to further increase the presence of rare vocabulary in the training data, we implement a sample selection strategy based on euphemism frequency and predictive uncertainty, integrating it with the earlier steps. Experimental results demonstrate that our framework outperforms previous research on most metrics and achieves a 10% increase over the baseline model.