Biomedical Named Entity Recognition (BioNER) is a key task in biomedical text processing and plays an important role in clinical decision support, drug research and development, and medical knowledge graph construction. Most traditional BioNERs use models trained for a single entity type. Although it performs well in a single specific field, it struggles to handle the complex task scenarios involving multi-entity type coexistence, uneven distribution of training data and heterogeneous labels. For this reason, we propose the KFC-Net framework that integrates knowledge distillation and cross-type entity recognition, which significantly improves the accuracy and deployment efficiency of the model in cross-type entity recognition tasks by adjusting the training strategy and lightweighting the model architecture. KFC-Net constructs a unified label space to solve the problem of predictive label conflicts in cross-type entity recognition, and applies a probabilistic aggregation strategy relying on the independence assumption to integrate the output distributions, ultimately training a student model with cross-type entity recognition capabilities. In the experimental validation of the three major tasks of diseases, chemical substances and genes, the F1-scores of KFC-Net ranked among the top, outperforming the traditional merging models and the mainstream baseline models. The framework effectively solves the label conflict problem in cross-type entity recognition and supports various lightweight student model architectures such as RoBERTa-base, DistilBERT, TinyBERT, etc. It can be flexibly selected according to the various hardware environment requirements to improve the feasibility of actual deployment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cross-Type Biomedical Named Entity Recognition Method Based on Knowledge Distillation

  • Zhengyao Wang,
  • JingYan Wang,
  • Tingwei Geng,
  • YanFang Wang,
  • Chi Yuan

摘要

Biomedical Named Entity Recognition (BioNER) is a key task in biomedical text processing and plays an important role in clinical decision support, drug research and development, and medical knowledge graph construction. Most traditional BioNERs use models trained for a single entity type. Although it performs well in a single specific field, it struggles to handle the complex task scenarios involving multi-entity type coexistence, uneven distribution of training data and heterogeneous labels. For this reason, we propose the KFC-Net framework that integrates knowledge distillation and cross-type entity recognition, which significantly improves the accuracy and deployment efficiency of the model in cross-type entity recognition tasks by adjusting the training strategy and lightweighting the model architecture. KFC-Net constructs a unified label space to solve the problem of predictive label conflicts in cross-type entity recognition, and applies a probabilistic aggregation strategy relying on the independence assumption to integrate the output distributions, ultimately training a student model with cross-type entity recognition capabilities. In the experimental validation of the three major tasks of diseases, chemical substances and genes, the F1-scores of KFC-Net ranked among the top, outperforming the traditional merging models and the mainstream baseline models. The framework effectively solves the label conflict problem in cross-type entity recognition and supports various lightweight student model architectures such as RoBERTa-base, DistilBERT, TinyBERT, etc. It can be flexibly selected according to the various hardware environment requirements to improve the feasibility of actual deployment.