In natural language processing, domain-specific summarization has its own unique challenge: it requires extracting the most domain-relevant content to the target from a given text. Additionally, in this scenario, the high compression ratios necessitated by over-length texts demand a model capable of precisely capturing pertinent parts. To alleviate these issues, we propose a two-stage enhanced abstractive summarization method for tailoring domain-specific summarization, transcending the conventional paradigms of single extractive or generative models. Specifically, since it is hard to determine the domain-specific content in the source text without fine-grained annotation, we first propose a domain-aligner matching algorithm in the initial stage, where source text sentences are pseudo-labeled by target abstractive summary sentences according to the correlation between them. Then, we design a hierarchical extractive summarization model HI-XLNET to deal with over-length documents, which could carefully select key sentences resonating with the target domain from both the token level and sentence level. In the second stage, we further use a generative summarization model for the refinement of the extracted summaries from the first stage to ensure coherence and conciseness. We especially select the higher-quality samples from the extractive model to train the abstractive model, ensuring the two models can work seamlessly together to produce a better final summary. The experimental result shows that our model has a significant superiority over several robust baselines, including ChatGPT, in two legal-related datasets (CAIL 2022 and Billsum), achieving the 1st rank in the public Chinese AI and Law Challenge (CAIL 2022).

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Tailored Domain-Specific Summaries: A Two-Stage Method Combining Extractive and Abstractive Summarization Models

  • Feng Jiang,
  • Lingyi Yang,
  • Yu Lu,
  • Haizhou Li

摘要

In natural language processing, domain-specific summarization has its own unique challenge: it requires extracting the most domain-relevant content to the target from a given text. Additionally, in this scenario, the high compression ratios necessitated by over-length texts demand a model capable of precisely capturing pertinent parts. To alleviate these issues, we propose a two-stage enhanced abstractive summarization method for tailoring domain-specific summarization, transcending the conventional paradigms of single extractive or generative models. Specifically, since it is hard to determine the domain-specific content in the source text without fine-grained annotation, we first propose a domain-aligner matching algorithm in the initial stage, where source text sentences are pseudo-labeled by target abstractive summary sentences according to the correlation between them. Then, we design a hierarchical extractive summarization model HI-XLNET to deal with over-length documents, which could carefully select key sentences resonating with the target domain from both the token level and sentence level. In the second stage, we further use a generative summarization model for the refinement of the extracted summaries from the first stage to ensure coherence and conciseness. We especially select the higher-quality samples from the extractive model to train the abstractive model, ensuring the two models can work seamlessly together to produce a better final summary. The experimental result shows that our model has a significant superiority over several robust baselines, including ChatGPT, in two legal-related datasets (CAIL 2022 and Billsum), achieving the 1st rank in the public Chinese AI and Law Challenge (CAIL 2022).