Named Entity Recognition (NER) aims to extract and label specific entity types from a given text. It is a crucial step in information extraction, yet certain specialised domains often face the challenge of insufficient training data. Most existing approaches focus primarily on using prompt-based learning methods, leveraging the knowledge inherent in pre-trained models to predict entity types. However, these methods often overlook the role of soft prompts in template construction. Soft prompts can offer a model with more specific context or guidance during training, aiding its understanding and language generation. In this regard, we propose a hybrid prompt design that integrates hard and soft prompts into our prompt template to improve performance (F1-score). Specifically, we reformulate the NER task as a text generation problem by using hard prompts and enriching semantics with a machine reading comprehension (MRC) query as a prefix hard prompt. Additionally, we design a soft prompt generator to automatically create soft prompts for each instance and defer the generation of soft prompts to the intermediate layers of the model rather than the input layer, thereby avoiding the loss of information caused by long propagation paths of soft prompts. The experimental results demonstrate the effectiveness of our proposed method in the few-shot task. Compared to previous methods, our model significantly improves performance, especially achieving a maximum improvement of 8.08 F1-score in the 10-shot setting.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Hybrid Prompt Method for Few-Shot Named Entity Recognition

  • Yuxiang Li,
  • Shmuel Tyszberowicz,
  • Zhiming Liu,
  • Bo Liu

摘要

Named Entity Recognition (NER) aims to extract and label specific entity types from a given text. It is a crucial step in information extraction, yet certain specialised domains often face the challenge of insufficient training data. Most existing approaches focus primarily on using prompt-based learning methods, leveraging the knowledge inherent in pre-trained models to predict entity types. However, these methods often overlook the role of soft prompts in template construction. Soft prompts can offer a model with more specific context or guidance during training, aiding its understanding and language generation. In this regard, we propose a hybrid prompt design that integrates hard and soft prompts into our prompt template to improve performance (F1-score). Specifically, we reformulate the NER task as a text generation problem by using hard prompts and enriching semantics with a machine reading comprehension (MRC) query as a prefix hard prompt. Additionally, we design a soft prompt generator to automatically create soft prompts for each instance and defer the generation of soft prompts to the intermediate layers of the model rather than the input layer, thereby avoiding the loss of information caused by long propagation paths of soft prompts. The experimental results demonstrate the effectiveness of our proposed method in the few-shot task. Compared to previous methods, our model significantly improves performance, especially achieving a maximum improvement of 8.08 F1-score in the 10-shot setting.