Parameter Efficient Fine-Tuned Vision LLMs for Syntax Aware Mathematical Equation Image to LaTeX Conversion
摘要
Automatic conversion of mathematical equation images into LaTeX code has emerged as a crucial technology for digitizing and preserving mathematical knowledge. This paper presents a robust end-to-end pipeline that combines syntax-aware data augmentation, advanced multi-stage image preprocessing, prompt engineering, model quantization with parameter-efficient large language model fine-tuning, and iterative post-processing refinement to improve equation generation accuracy. Leveraging a large-scale dataset, we systematically evaluate state-of-the-art vision large language models, including Qwen2-VL-7B, Llama-3.2-11B-ViT, and Gemma3-4B. We have evaluated our pipeline across BLEU, Levenshtein normalized edit distance and exact match metrics. The Qwen2 model exhibits substantial improvements when enhanced through fine-tuning and iterative refinement. Detailed error analysis reveals remaining challenges with visually similar symbols, ambiguous character pairs and complex layouts. Our findings highlight persistent bottlenecks and outline future research directions in targeted augmentation, higher-resolution processing, and model architecture improvements. The proposed pipeline thus provides a scalable, accurate and adaptable solution for real-world mathematical document understanding and digital transformation.