Recently, pre-trained language models (PLMs) have achieved state-of-the-art performance in various natural language processing (NLP) tasks. However, it has been proven that PLMs lack knowledge in tasks such as named entity recognition (NER). Despite many efforts to inject knowledge into PLMs, this problem remains unresolved. The currently best-performing models mainly include knowledge graphs (KG) and knowledge prompting models. Unfortunately, the former is not easily obtainable and consumes computational resources, while the latter requires a large amount of storage space for the prompting context. This limits their application in many industrial fields that require real-time NER responses. In this work, we propose a simple and effective vocabulary enhancement method (ESTOR) that adds prior knowledge to the model through enumerative matching. This approach avoids the need for designing complex sequence modeling architectures and wasting computational and storage resources. This reduces memory and graphics memory usage by approximately 50% compared to other models. Additionally, the proposed method can be easily combined with pre-trained models such as BERT. Achieved the best results on four benchmark Chinese NER datasets, demonstrating that ESTOR can resolve ambiguities while integrating rich information from geographical dictionaries. Moreover, ESTOR exhibits strong applicability in low-resource domains and industrial scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ESTOR: Enumerate-Specify-Tutor Mechanism Used of Lexicon in Chinese NER

  • Binghong Wu,
  • Maihemuti Maimaiti,
  • Xian Yi,
  • Gulinigeer Abudouwaili

摘要

Recently, pre-trained language models (PLMs) have achieved state-of-the-art performance in various natural language processing (NLP) tasks. However, it has been proven that PLMs lack knowledge in tasks such as named entity recognition (NER). Despite many efforts to inject knowledge into PLMs, this problem remains unresolved. The currently best-performing models mainly include knowledge graphs (KG) and knowledge prompting models. Unfortunately, the former is not easily obtainable and consumes computational resources, while the latter requires a large amount of storage space for the prompting context. This limits their application in many industrial fields that require real-time NER responses. In this work, we propose a simple and effective vocabulary enhancement method (ESTOR) that adds prior knowledge to the model through enumerative matching. This approach avoids the need for designing complex sequence modeling architectures and wasting computational and storage resources. This reduces memory and graphics memory usage by approximately 50% compared to other models. Additionally, the proposed method can be easily combined with pre-trained models such as BERT. Achieved the best results on four benchmark Chinese NER datasets, demonstrating that ESTOR can resolve ambiguities while integrating rich information from geographical dictionaries. Moreover, ESTOR exhibits strong applicability in low-resource domains and industrial scenarios.