Scholarlink

Abstract <p>Despite recent progress, Optical Character Recognition (OCR) on historical newspapers still leaves 5–10% character errors. We present a fully automated post-OCR normalization pipeline that combines lightweight 7–8B instruction-tuned LLMs quantized to 4-bit (INT4) with a small set of regex rules. On the BLN600 benchmark (600 pages of 19th-century British newspapers), our best model YandexGPT-5-Instruct Q4 reduces the Character Error Rate (CER) from 8.4 to 4.0% (–52.5%) and the Word Error Rate (WER) from 20.2 to 6.5% (–67.8%), while raising semantic similarity to 0.962. The system runs on consumer hardware (RTX-4060 Ti and 8 GB VRAM) at about 35 s per page and requires no fine-tuning or parallel training data. These results indicate that compact INT4 LLMs are a practical alternative to large checkpoints for post-OCR cleanup of historical documents.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Normalization of Text Recognized by Optical Character Recognition Using Lightweight LLMs

  • 期刊论文

  • 发表日期：

    2025年12月1日

查看全文
  • V. K. Vershinin,
  • I. V. Khodnenko,
  • S. V. Ivanov

摘要

Abstract

Despite recent progress, Optical Character Recognition (OCR) on historical newspapers still leaves 5–10% character errors. We present a fully automated post-OCR normalization pipeline that combines lightweight 7–8B instruction-tuned LLMs quantized to 4-bit (INT4) with a small set of regex rules. On the BLN600 benchmark (600 pages of 19th-century British newspapers), our best model YandexGPT-5-Instruct Q4 reduces the Character Error Rate (CER) from 8.4 to 4.0% (–52.5%) and the Word Error Rate (WER) from 20.2 to 6.5% (–67.8%), while raising semantic similarity to 0.962. The system runs on consumer hardware (RTX-4060 Ti and 8 GB VRAM) at about 35 s per page and requires no fine-tuning or parallel training data. These results indicate that compact INT4 LLMs are a practical alternative to large checkpoints for post-OCR cleanup of historical documents.