While Large Language Models (LLMs) have demonstrated remarkable performance in Natural Language Processing (NLP) tasks, their application in specialized domains still faces challenges such as data scarcity and difficulty in constructing domain data. Existing text chunking methods often struggle to balance semantic coherence with structural relationships when processing complex documents containing technical texts, tables, formulas, and images. In this paper, we propose the Semantic-Structural Synergistic Chunking Strategy, which addresses semantic fragmentation and structural imbalance in traditional approaches through dynamic identification of semantic units and reconstruction of document organization. Specifically, our method integrates the semantic comprehension capabilities of LLMs with multiple document parsing technologies, employing bidirectional feedback to ensure that chunk boundaries preserve both semantic consistency and global logical structure. Experimental results on military aircraft design manual demonstrate that our strategy significantly outperforms baseline methods in BLEU and ROUGE evaluation metrics, validating its effectiveness in refining domain-specific datasets for LLMs. This study not only advances text chunking strategies but also provides new perspectives and frameworks for optimizing vertical-domain LLMs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Semantic-Structural Synergistic Chunking Method for Enhancing Domain-Specific LLM Fine-Tuning

  • Baosheng Yin,
  • Dapeng Li,
  • Zhenlai Xu,
  • Naiyu Hu

摘要

While Large Language Models (LLMs) have demonstrated remarkable performance in Natural Language Processing (NLP) tasks, their application in specialized domains still faces challenges such as data scarcity and difficulty in constructing domain data. Existing text chunking methods often struggle to balance semantic coherence with structural relationships when processing complex documents containing technical texts, tables, formulas, and images. In this paper, we propose the Semantic-Structural Synergistic Chunking Strategy, which addresses semantic fragmentation and structural imbalance in traditional approaches through dynamic identification of semantic units and reconstruction of document organization. Specifically, our method integrates the semantic comprehension capabilities of LLMs with multiple document parsing technologies, employing bidirectional feedback to ensure that chunk boundaries preserve both semantic consistency and global logical structure. Experimental results on military aircraft design manual demonstrate that our strategy significantly outperforms baseline methods in BLEU and ROUGE evaluation metrics, validating its effectiveness in refining domain-specific datasets for LLMs. This study not only advances text chunking strategies but also provides new perspectives and frameworks for optimizing vertical-domain LLMs.