Named Entity Recognition Using EHealth-BiLSTM-CRF Combine with Multi-head Self-attention for Chinese Medical Information

This research investigates how to extract crucial information from large amounts of medical data in an effective and precise manner. The BERT pre-training language model has become a popular tool for named entity recognition techniques. However, the method of randomly masking a single token used in the BERT pre-training strategy does not fully utilize the training data’s lexical, syntactic, and semantic structure for modeling. As a result, BERT is not a suitable solution for Chinese entity identification, so this research employs Ernie-Health (a Chinese language representation model pre-trained over large-scale biomedical text corpora) as the model’s embedding layer, adds a multi-head self-attention layer, and presents an EHealth-BiLSTM-Attention-CRF model that focuses on medical entity recognition. The F1 value of the model provided in this study for the three datasets CMeEE, Yidu-S4K, and cMedQANER is 78.34%, 90.24%, and 86.62%, respectively. The experimental results reveal that the proposed method outperformed other traditional methods in the task of named entity recognition in the Chinese medical field.