A Chinese Named Entity Recognition Approach with Multi-head Attention

Named entity recognition is the basis of many downstream tasks in the field of natural language processing, and its accuracy directly affects the work effect of subsequent tasks. In order to solve the problems of insufficient feature extraction ability and difficulty in capturing long-distance dependence of existing Chinese named entity recognition models, we propose a Chinese named entity recognition method that integrates multi-head attention and bidirectional gated recurrent unit. Firstly, the ALBERT pre-trained model is used to generate dynamic word vectors, and the vector sequences are input into BiGRU to extract global semantic features, and then the multi-head attention mechanism is used to capture long-distance dependent information to enhance semantic features, and finally the optimal sequence is obtained by conditional random field decoding. Experiments show that the proposed method achieves 95.84% and 94.93% F1 on the People’s Daily and MSRA Chinese datasets, respectively, which is better than other models, and the training time of the proposed method is reduced by about 14% compared with the BERT-BiLSTM-CRF model, which proves the effectiveness and versatility of our model.