Semi-End-to-End Nested Named Entity Recognition from Speech

There are two approaches for Named Entity Recognition (NER) from speech: two-step pipeline and End-to-End (E2E). In the pipeline approach, cascading errors are inevitable. In the E2E approach, its annotation method poses a challenge to Automatic Speech Recognition (ASR) when Named Entities (NEs) are nested. This is because multiple special tokens without audio signals between words will exist, which may even cause ambiguity problems for NER. In this paper, we propose a new paradigm and name it semi-E2E, as it completes parts of NER in ASR. Specifically, we introduce a novel annotation method for nested NEs, where only two special tokens are used to annotate the heads (The head of an NE is its first word, for examples, “western” is the head of the NE “western Canadian” in Fig. 1. If an NE has only one word, its head is itself.) of NEs, regardless of the number of NE categories. Also, we use a span classifier to classify only the spans that start with the predicted heads in transcriptions. From the experimental results on the nested NER dataset of Chinese speech CNERTA, our semi-E2E approach gets the best F1 score (1.84% and 0.53% absolute points higher than E2E and pipeline respectively).