Scholarlink

<p>Language models often underperform when fine tuned with limited labelled data. To reduce dependence on costly annotation, recent studies have explored clustering and pseudo labelling to leverage unlabeled text with only a few labelled examples for fine tuning. However, these methods face persistent challenges: clustering errors and pseudo-label mismatches can degrade fine-tuning performance. We propose <i>Cluster-SCP</i>, a novel framework that improves intra-cluster coherence and reduces cluster quantity to generate more reliable pseudo labels. The framework begins with K-Means initialization and applies an iterative refinement process, <i>Intermediate Pseudo Labels</i>, comprising two stages: embedding reassignment to minimize clustering errors, and cluster merging via contrastive learning with graph readouts. The refined pseudo labels are first used to fine tune language model, which is subsequently refined using a small set of labels. Experiments on three benchmark datasets show that Cluster-SCP consistently outperforms baseline and state-of-the-art methods, achieving at least a 5.3% accuracy improvement while reducing the number of clusters by 22%.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cluster-SCP: Similarity and Contrastive Learning to Enhance Pseudo Labels for Fine Tuning under Few Labels

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月1日

查看全文
  • Abdullah Alsuhaibani,
  • Abdulrahman Alalawi,
  • Imran Razzak,
  • Shoaib Jameel,
  • Xianzhi Wang,
  • Guandong Xu

摘要

Language models often underperform when fine tuned with limited labelled data. To reduce dependence on costly annotation, recent studies have explored clustering and pseudo labelling to leverage unlabeled text with only a few labelled examples for fine tuning. However, these methods face persistent challenges: clustering errors and pseudo-label mismatches can degrade fine-tuning performance. We propose Cluster-SCP, a novel framework that improves intra-cluster coherence and reduces cluster quantity to generate more reliable pseudo labels. The framework begins with K-Means initialization and applies an iterative refinement process, Intermediate Pseudo Labels, comprising two stages: embedding reassignment to minimize clustering errors, and cluster merging via contrastive learning with graph readouts. The refined pseudo labels are first used to fine tune language model, which is subsequently refined using a small set of labels. Experiments on three benchmark datasets show that Cluster-SCP consistently outperforms baseline and state-of-the-art methods, achieving at least a 5.3% accuracy improvement while reducing the number of clusters by 22%.