Exploring Named Entity Recognition in Medical Knowledge Graphs with Pre-trained Language Models and Attention Mechanism

Named entity recognition plays a crucial role in natural language processing and directly impacts the performance of downstream tasks. Pre-trained models have become a groundbreaking advancement in artificial intelligence and are commonly employed for NER tasks. However, when dealing with medical-related tasks, utilizing a pre-trained model trained on general corpora may lead to a significant decline in performance. To address this issue, we propose a neural network called the “Named Entity Recognition Model Integrated with Medical Knowledge Graph.” This model builds upon previous research in medical named entity recognition tasks. In the knowledge representation phase, our model incorporates a knowledge graph by employing relative position encoding. This inclusion enhances the pre-trained model’s ability to capture domain-specific knowledge and mitigates the loss of semantic and structural information that typically occurs when incorporating triplets. Additionally, a multi-head attention layer aligns the knowledge graph with semantic features, enabling the model to learn the semantic associations between the two and aiding the model in making accurate annotation decisions. Experimental results demonstrate that our model outperforms other mainstream named entity recognition models. On the diabetes and CCKS2017 datasets, our model achieves an absolute increase of 4.8% and 2.13% in the F1 value compared to the baseline model, confirming the superior performance of our proposed model. Furthermore, we conduct corresponding ablation experiments to showcase the effectiveness of our innovation.