The scarcity of high-quality image-text paired data severely constrains the development of multimodal large models in complex visual semantic understanding. Existing public datasets commonly suffer from noise introduced by web crawling, lack of domain-specific semantic annotations, and weak logical alignment between images and text, rendering them inadequate for interdisciplinary research. This paper proposes SciData-Factory, an automated framework for generating image-text datasets from academic literature to construct high-quality cross-modal data. The framework integrates a proprietary PDF parsing engine with a YOLO-based layout detection model and a large language model (e.g., Qwen-long) to achieve precise alignment between figures and their corresponding descriptive text in scholarly documents, and implements human-machine collaborative optimization strategies to enhance data quality. Experimental results demonstrate that datasets constructed using this framework significantly improve image understanding performance of multimodal large models in fine-tuning tasks across domains such as satellite imagery and agriculture.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SciData-Factory: An Automated Framework for Generating High-Quality Image-Text Datasets

  • Li Zhu,
  • Shaohua Li,
  • Huan Li,
  • Qianqian Lian,
  • Siao Yu,
  • Hang Chen

摘要

The scarcity of high-quality image-text paired data severely constrains the development of multimodal large models in complex visual semantic understanding. Existing public datasets commonly suffer from noise introduced by web crawling, lack of domain-specific semantic annotations, and weak logical alignment between images and text, rendering them inadequate for interdisciplinary research. This paper proposes SciData-Factory, an automated framework for generating image-text datasets from academic literature to construct high-quality cross-modal data. The framework integrates a proprietary PDF parsing engine with a YOLO-based layout detection model and a large language model (e.g., Qwen-long) to achieve precise alignment between figures and their corresponding descriptive text in scholarly documents, and implements human-machine collaborative optimization strategies to enhance data quality. Experimental results demonstrate that datasets constructed using this framework significantly improve image understanding performance of multimodal large models in fine-tuning tasks across domains such as satellite imagery and agriculture.