Tailored Domain-Specific Summaries: A Two-Stage Method Combining Extractive and Abstractive Summarization Models
摘要
In natural language processing, domain-specific summarization has its own unique challenge: it requires extracting the most domain-relevant content to the target from a given text. Additionally, in this scenario, the high compression ratios necessitated by over-length texts demand a model capable of precisely capturing pertinent parts. To alleviate these issues, we propose a two-stage enhanced abstractive summarization method for tailoring domain-specific summarization, transcending the conventional paradigms of single extractive or generative models. Specifically, since it is hard to determine the domain-specific content in the source text without fine-grained annotation, we first propose a domain-aligner matching algorithm in the initial stage, where source text sentences are pseudo-labeled by target abstractive summary sentences according to the correlation between them. Then, we design a hierarchical extractive summarization model HI-XLNET to deal with over-length documents, which could carefully select key sentences resonating with the target domain from both the token level and sentence level. In the second stage, we further use a generative summarization model for the refinement of the extracted summaries from the first stage to ensure coherence and conciseness. We especially select the higher-quality samples from the extractive model to train the abstractive model, ensuring the two models can work seamlessly together to produce a better final summary. The experimental result shows that our model has a significant superiority over several robust baselines, including ChatGPT, in two legal-related datasets (CAIL 2022 and Billsum), achieving the 1st rank in the public Chinese AI and Law Challenge (CAIL 2022).