A financial table structure recognition method based on transformer with attention enhancement
摘要
In the financial domain, table structure recognition is vital for automating financial information processing and enhancing decision-making efficiency. Tables often contain complex structures, such as with or without borders and spanning across rows and columns, as well as visual disturbances like background colors, which can lead to sequence prediction errors. Firstly, to address issues such as sticky cells and over segmentation, in this paper, we propose a novel collaborative attention module to improve the model’s perception of local details such as cell boundaries. Secondly, to address the issue of boundary drift caused by using the Pix2Seq method in the cell boundary detection task, we introduce the Mamba model to selectively capture the underlying patterns of cell position variations through its Selective State Space Modeling, which improves the precision of cell boundary detection. Meanwhile, we propose a novel positional loss function to further enhance the model’s sensitivity to the spatial positioning of cells. Furthermore, to address the scarcity of Chinese financial table data, we construct the Chinese financial table dataset from real financial PDFs, which includes more complex table structures and variations in background colors. Experiments have demonstrated that our method performs better in table structure recognition, especially in financial table structure recognition. Our method achieves mAP and S-TEDS values of 74.95% and 95.17% on the PubTabNet dataset, respectively. After fine-tuning on the Chinese financial table dataset, the mAP score improved to 66.70%. Our code is publicly available at: https://github.com/LeKit089/FinTSR.