EntityLayout: Entity-Level Pre-training Language Model for Semantic Entity Recognition and Relation Extraction

Semantic entity recognition (SER) and relation extraction (RE) are the core tasks of information extraction from visually-rich documents (VrDs). Although self-supervised pre-training models have advanced the performance of these tasks, existing methods are insufficient in fusing the token features and modeling SER and RE jointly. In this paper, we propose an entity-level language model, named EntityLayout, and use a graph-based approach for efficient entity labeling and linking jointly on a continual pre-training model. In EntityLayout, a Token Fusion Module (TFM) is proposed to fuse the token feature and learn an entity-level representation. Then, we use the entity-level representation to build a graph and propose a Graph Pruning Module (GPM) to effectively prune the invalid links. Finally, SER and RE are accomplished simultaneously by a joint information extraction Task Module. Experimental results on public datasets FUNSD and CORD demonstrate that the proposed EntityLayout achieves competitive performance in SER and state-of-the-art performance in RE, i.e., SER F1 scores of 0.9108 and 0.9650, respectively, RE F1 scores of 0.8212 and 0.9898, respectively.