Recognizing and rearranging handwritten documents is essential for computer-aided information retrieval, digitization, and archive systems. The intricacy of handwritten documents is not adequately addressed by traditional optical character recognition (OCR) systems due to variations in handwriting styles, asymmetrical page layouts, and subpar document quality. This work targets page-level recognition and reordering of handwritten text with the ICDAR 2024 Handwriting Text Recognition (HTR) dataset, consisting of 15,000 pages of few Indic languages with intricate layouts and reading orders. We evaluate two state-of-the-art models for Handwritten Text Recognition (HTR). Firstly, Llama 3.2-Vision is a Vision Transformer model capable of performing end-to-end paragraph-level recognition without explicit segmentation. The second model is Paddle OCR. It is an open-source CNN-RNN-CTC-based model that relies online segmentation but struggles with cursive and poor-quality handwriting. Experimental findings reveal that Llama 3.2-Vision performs much better than Paddle OCR with a Character Error Rate (CER) of 0.01 and a Word Error Rate (WER) of 0.07, while Paddle OCR has a CER of 0.44 and a WER of 0.65. For document reordering, we use two methods: a semantic similarity-based approach employing Sentence Transformer embeddings, achieving a reordering accuracy (RA) of 73.20%, and an enhanced hybrid reordering method that combines geometric layout characteristics with graph-based optimization using Sinkhorn normalization, improving RA to 91.5%. This work emphasizes the advantage of transformer-based recognition models and the success of hybrid reordering methods in dealing with intricate handwritten document structure.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhanced Semantic Approach for Reordering of Handwritten Documents

  • S. Ravichandran,
  • R. Kasturi Rangan,
  • H. R. Nandish,
  • S. Karthik,
  • T. N. Hemanth

摘要

Recognizing and rearranging handwritten documents is essential for computer-aided information retrieval, digitization, and archive systems. The intricacy of handwritten documents is not adequately addressed by traditional optical character recognition (OCR) systems due to variations in handwriting styles, asymmetrical page layouts, and subpar document quality. This work targets page-level recognition and reordering of handwritten text with the ICDAR 2024 Handwriting Text Recognition (HTR) dataset, consisting of 15,000 pages of few Indic languages with intricate layouts and reading orders. We evaluate two state-of-the-art models for Handwritten Text Recognition (HTR). Firstly, Llama 3.2-Vision is a Vision Transformer model capable of performing end-to-end paragraph-level recognition without explicit segmentation. The second model is Paddle OCR. It is an open-source CNN-RNN-CTC-based model that relies online segmentation but struggles with cursive and poor-quality handwriting. Experimental findings reveal that Llama 3.2-Vision performs much better than Paddle OCR with a Character Error Rate (CER) of 0.01 and a Word Error Rate (WER) of 0.07, while Paddle OCR has a CER of 0.44 and a WER of 0.65. For document reordering, we use two methods: a semantic similarity-based approach employing Sentence Transformer embeddings, achieving a reordering accuracy (RA) of 73.20%, and an enhanced hybrid reordering method that combines geometric layout characteristics with graph-based optimization using Sinkhorn normalization, improving RA to 91.5%. This work emphasizes the advantage of transformer-based recognition models and the success of hybrid reordering methods in dealing with intricate handwritten document structure.