<p>Pre-training language models are increasingly being utilized for multi-document summarization (MDS) tasks. However, pre-training typically necessitates large-scale, domain-specific data. Most MDS pre-training models process multiple documents as a single document, while ignoring the subsection relationship information among different documents. In this paper, we focus on pre-training objectives for MDS, which assume key information appears across multiple documents that point to the same topic. We segment each document into several subsections based on text structure features. Then, we compare the subsections of different documents, extract key sentences from these subsections using text similarity, and generate a proxy summary. Experimental results on Multi-News and WikiSum demonstrate that our proposed model outperforms compared MDS models in terms of ROUGE scores and maintains strong performance even with limited data samples.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-document summarization through subsection-aware pre-training objectives

  • Xianchuan Wang,
  • Ben Lu,
  • Wenkai Ming,
  • Xianchao Wang

摘要

Pre-training language models are increasingly being utilized for multi-document summarization (MDS) tasks. However, pre-training typically necessitates large-scale, domain-specific data. Most MDS pre-training models process multiple documents as a single document, while ignoring the subsection relationship information among different documents. In this paper, we focus on pre-training objectives for MDS, which assume key information appears across multiple documents that point to the same topic. We segment each document into several subsections based on text structure features. Then, we compare the subsections of different documents, extract key sentences from these subsections using text similarity, and generate a proxy summary. Experimental results on Multi-News and WikiSum demonstrate that our proposed model outperforms compared MDS models in terms of ROUGE scores and maintains strong performance even with limited data samples.