DocAnnot - Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation
摘要
Key Information Extraction (KIE) from documents is a crucial task in many applications, but the creation of training datasets for KIE models is traditionally a time-consuming and expensive manual process. In this paper we introduce a novel Framework DocAnnot that significantly accelerates and reduces the cost of KIE dataset generation. Our framework leverages a Large Vision Language Model (LVLM) for label value extraction, Optical Character Recognition (OCR) for text and bounding box detection, and a novel Spatially Informed Contextual Matching (SICM) algorithm to accurately associate extracted values with their corresponding labels. SICM enhances label-value association by incorporating spatial relationships and proximity analysis alongside textual matching. We evaluate our framework on two benchmark KIE datasets, CORD and SROIE, demonstrating its ability to automatically generate annotations with reasonable accuracy (achieving F1-scores of 0.679 and 0.846). Furthermore, we investigate the effectiveness of using the auto-annotated data for fine-tuning downstream KIE models. While models trained on human-annotated data achieve superior performance, our experiments show that models trained exclusively on auto-annotated data still attain respectable performance levels (e.g., LayoutLMv3 achieving an F1-score of 0.6765 on CORD). These results demonstrate that while our framework significantly reduces reliance on manual annotation, it does not yet fully eliminate the need for human intervention. However, by automating the annotation process to a point where human reviewers can efficiently verify and refine the outputs, our system enables near-perfect annotations with far greater efficiency than manual annotation from scratch. This approach offers substantial time and cost savings while optimizing the balance between automation and accuracy, making it particularly valuable for resource-constrained settings and rapid model prototyping.