A Fusion Tuning Method for Named Entity Recognition
摘要
In named entity recognition, the main methods for constructing deep neural networks are fine-tuning and prompt tuning. Fine-tuning is a commonly used paradigm to optimize neural networks by using task-specific objective functions to make them more adapted to solve downstream tasks. In which pre-trained language models are used as external resources to provide contextual representations. Prompt tuning is a recent paradigm in which the downstream tasks are reformulated to look more like those solved during the original PLM training with the help of a textual prompt. It is effective to utilize potential knowledge of PLMs. In this paper, we propose the fusion approach, which sufficiently fuses the semantics obtained by extracting fine-tuning and prompt-tuning. In our approach, the model makes decisions based on the information obtained from the reconstructive fusion of the fine-tuned semantics and the semantics of the pre-trained language model. The representation obtained by fine-tuning and the representation obtained by prompt tuning are reconstructed into a multidimensional semantics using a gate fusion module (GFM). In particularly, our model has two gate components, one gate to extract the semantics of fine-tuning and the other gate to extract the semantics of prompt tuning, and finally the reconstructed semantic representation is obtained. This takes full advantage of the potential knowledge of PLM. Our model is evaluated on the CONLL2003, ACE2005 and GENIA corpora. It achieves performance close to the state-of-the-art in F1 scores.