Scholarlink

<p>Active prompt learning using vision-language models (<i>e.g.</i>, CLIP) has gained attention for reducing reliance on large-scale labeled datasets in downstream task adaptation. Existing methods mechanically annotate query set and retrain the model in each round until the labeling budget is exhausted. However, these methods often overlook the difficulty correlations among query sets across rounds, leading to the premature selection of highly informative yet excessively complex samples, which in turn hampers effective model optimization. In addition, they disregard the diverse and complementary information embedded within the remaining unlabeled data, further limiting the model’s ability to develop comprehensive and robust feature representations. To address these limitations, we propose a novel Discriminative Self-Training Dual-Curriculum Learning (SDC) approach, which enhances active prompt learning by hierarchically constructing two easy-to-hard curriculums across round and batch dimensions, while selectively incorporating reliable pseudo-labeled data based on the confidence and uncertainty of average prediction. Specifically, SDC consists of two core components: Dual-Curriculum Learning (Dual-CL) and CLIP-based Discriminative Self-Training (CDST) mechanism. Dual-CL integrates Round-to-Round Curriculum Learning (R2R-CL) and Batch-to-Batch Curriculum Learning (B2B-CL) to hierarchically adjusting data difficulty from round and batch dimensions, thereby mitigating the impact of excessively complex samples and enhancing the model’s adaptability to various data. Firstly, R2R-CL employs a transfer teacher to evaluate the difficulty of unlabeled samples and designs a Dynamic Gaussian Threshold strategy to select those below an adaptive round threshold for real-labeled data generation. As this threshold increases over successive rounds, progressively harder samples are incorporated, allowing the student CLIP to gradually acquire a deeper understanding of data structure and characteristics across rounds. Meanwhile, CDST leverages the teacher CLIP to generate reliable pseudo-labeled data from the remaining unlabeled data, which integrates with real-labeled data to facilitate the student CLIP’s prompt learning. Given that the student CLIP initially struggles to train with extremely difficult samples from labeled and unlabeled data, B2B-CL dynamically adjusts their contribution to Batch-to-Batch loss by initially down-weighting and subsequently up-weighting their influence during training. This facilitates an easy-to-hard prompt learning process, thereby mitigating training instability and slow convergence. Extensive experiments conducted on different real-world datasets demonstrate the effectiveness of the proposed SDC.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Boosting Active Prompt Learning via Discriminative Self-Training Dual-Curriculum Learning

  • 期刊论文

  • 发表日期：

    2026年1月10日

查看全文
  • Sen Tao,
  • Jiawei Liu,
  • Peng Zeng,
  • Yongchao Xu,
  • Bingyu Hu,
  • Zheng-Jun Zha

摘要

Active prompt learning using vision-language models (e.g., CLIP) has gained attention for reducing reliance on large-scale labeled datasets in downstream task adaptation. Existing methods mechanically annotate query set and retrain the model in each round until the labeling budget is exhausted. However, these methods often overlook the difficulty correlations among query sets across rounds, leading to the premature selection of highly informative yet excessively complex samples, which in turn hampers effective model optimization. In addition, they disregard the diverse and complementary information embedded within the remaining unlabeled data, further limiting the model’s ability to develop comprehensive and robust feature representations. To address these limitations, we propose a novel Discriminative Self-Training Dual-Curriculum Learning (SDC) approach, which enhances active prompt learning by hierarchically constructing two easy-to-hard curriculums across round and batch dimensions, while selectively incorporating reliable pseudo-labeled data based on the confidence and uncertainty of average prediction. Specifically, SDC consists of two core components: Dual-Curriculum Learning (Dual-CL) and CLIP-based Discriminative Self-Training (CDST) mechanism. Dual-CL integrates Round-to-Round Curriculum Learning (R2R-CL) and Batch-to-Batch Curriculum Learning (B2B-CL) to hierarchically adjusting data difficulty from round and batch dimensions, thereby mitigating the impact of excessively complex samples and enhancing the model’s adaptability to various data. Firstly, R2R-CL employs a transfer teacher to evaluate the difficulty of unlabeled samples and designs a Dynamic Gaussian Threshold strategy to select those below an adaptive round threshold for real-labeled data generation. As this threshold increases over successive rounds, progressively harder samples are incorporated, allowing the student CLIP to gradually acquire a deeper understanding of data structure and characteristics across rounds. Meanwhile, CDST leverages the teacher CLIP to generate reliable pseudo-labeled data from the remaining unlabeled data, which integrates with real-labeled data to facilitate the student CLIP’s prompt learning. Given that the student CLIP initially struggles to train with extremely difficult samples from labeled and unlabeled data, B2B-CL dynamically adjusts their contribution to Batch-to-Batch loss by initially down-weighting and subsequently up-weighting their influence during training. This facilitates an easy-to-hard prompt learning process, thereby mitigating training instability and slow convergence. Extensive experiments conducted on different real-world datasets demonstrate the effectiveness of the proposed SDC.