<p>Domain adaptation aims to facilitate learning tasks with no or few labels in the target domain by utilizing labeled data from the source domain. Recently, prompt-tuning has emerged as a powerful instrument in various downstream tasks including domain adaptation. However, due to that the prompt-tuning can achieve impressive performance even in the few-shot scenarios, most existing prompt-tuning methods neither take full advantage of the labeled information in the source domain nor the massive unlabeled information in the target domain. In this paper, we propose an innovative Cross-domain Prompt-tuning method for Domain Adaptation, which can consider both the labeled and unlabeled information in the source and target domains to construct and optimize prompt-tuning for minimizing the domain discrepancy. Specifically, the pre-trained language models (PLMs) is first fine-tuned based on the labeled data in the source domain, and then the top <i>N</i> synonyms related to the names of the category are retrieved by the fine-tuned PLMs. Meanwhile, we employ three different strategies to expand the label word space for modifying prompts based on the unlabeled data in the target domain. Finally, the intersection of both the source and target domains is adopted for the final prompt construction. Compared to existing methods that may lack comprehensiveness or contain ambiguous terms, our method has a broader coverage and more precise meanings prompt for domain adaptation. The extensive experimental results demonstrated that the obvious improvement is obtained compared with other state-of-the-art methods on three well-known domain adaptation benchmarks.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cross-domain prompt-tuning for domain adaptation

  • Shuqin Wang,
  • Yun Li,
  • Yi Zhu,
  • Jipeng Qiang,
  • Yunhao Yuan

摘要

Domain adaptation aims to facilitate learning tasks with no or few labels in the target domain by utilizing labeled data from the source domain. Recently, prompt-tuning has emerged as a powerful instrument in various downstream tasks including domain adaptation. However, due to that the prompt-tuning can achieve impressive performance even in the few-shot scenarios, most existing prompt-tuning methods neither take full advantage of the labeled information in the source domain nor the massive unlabeled information in the target domain. In this paper, we propose an innovative Cross-domain Prompt-tuning method for Domain Adaptation, which can consider both the labeled and unlabeled information in the source and target domains to construct and optimize prompt-tuning for minimizing the domain discrepancy. Specifically, the pre-trained language models (PLMs) is first fine-tuned based on the labeled data in the source domain, and then the top N synonyms related to the names of the category are retrieved by the fine-tuned PLMs. Meanwhile, we employ three different strategies to expand the label word space for modifying prompts based on the unlabeled data in the target domain. Finally, the intersection of both the source and target domains is adopted for the final prompt construction. Compared to existing methods that may lack comprehensiveness or contain ambiguous terms, our method has a broader coverage and more precise meanings prompt for domain adaptation. The extensive experimental results demonstrated that the obvious improvement is obtained compared with other state-of-the-art methods on three well-known domain adaptation benchmarks.