Enhancing inter-sentence coherence of extractive summarization with multitask learning
摘要
Extractive summarization is a widely applied technique in information extraction, information retrieval and natural language processing (NLP) applications. However, traditional methods often struggle to maintain inter-sentence coherence in the extracted summaries. This paper proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and a coherence discriminator module. The coherence discriminator is trained online using augmented sentence sequences. Its task is to judge whether the input sentences (embeddings) form a coherent sequence. Meanwhile, we maximize the coherent scores from the coherence discriminator by updating the parameters of the summarizer. Updating mechanisms of different modules by three learning objectives is designed for effective training. To make the sentence selection procedure differentiable for training, we apply a Gumbel-softmax function with a straight-through trick. Two representation merging strategies are introduced to ensure that the coherence discriminator always takes a sequence of sentence representations without masking vectors. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in extracted summaries based on their positions in the original article, while goodness in terms of relevance metrics (i.e., Rouge scores and BertScores) are preserved. In addition, model-based evaluation (UniEval) and human evaluation also evidence the significant improvement in coherence and consistency of the extracted summaries given by the enhanced extractive summarizer.