Improving Chinese word segmentation with character–lexicon class attention

Character–lexicon matching positions are important in Chinese word segmentation (CWS). Inspired by wordhood memory segmentation (WMSEG) and non-flat-lattice transformer (NFLAT), we consider character–lexicon attention with word match position and calculate the probability of characters appearing at the beginning (B), end (E), middle (M), and single (S) of words. We therefore propose a neural framework, CLSEG, which incorporates mask and class attention mechanisms to generate informative position features for characters, based on both character and lexicon contexts. The framework encodes character in base model (TENER) and pretrained models (ZEN, BERT-wwm and BABERT) with context feature and makes lexicon and tag fusion in attention. Compared with NFLAT, CLSEG changes lexicon values to character–tag values which are assigned weights from character–lexicon class attention. Compared with WMSEG, CLSEG decouples character–lexicon attention from NFLAT with multi-head and relative attention. Experimental studies incorporates YJ and LS lexicon for encoding. Experimental results and analyses on four benchmark CWS datasets demonstrate the superiority of CLSEG over existing approaches. The source code of the proposed method is publicly available at https://github.com/na978292231/CLSEG/tree/main/CLSSEG4CWS-main.