<p>Extractive summarization is a widely applied technique in information extraction, information retrieval and natural language processing (NLP) applications. However, traditional methods often struggle to maintain inter-sentence coherence in the extracted summaries. This paper proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and a coherence discriminator module. The coherence discriminator is trained online using augmented sentence sequences. Its task is to judge whether the input sentences (embeddings) form a coherent sequence. Meanwhile, we maximize the coherent scores from the coherence discriminator by updating the parameters of the summarizer. Updating mechanisms of different modules by three learning objectives is designed for effective training. To make the sentence selection procedure differentiable for training, we apply a Gumbel-softmax function with a straight-through trick. Two representation merging strategies are introduced to ensure that the coherence discriminator always takes a sequence of sentence representations without masking vectors. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in extracted summaries based on their positions in the original article, while goodness in terms of relevance metrics (<i>i.e.,</i> Rouge scores and BertScores) are preserved. In addition, model-based evaluation (UniEval) and human evaluation also evidence the significant improvement in coherence and consistency of the extracted summaries given by the enhanced extractive summarizer.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing inter-sentence coherence of extractive summarization with multitask learning

  • Renlong Jie,
  • Xiaojun Meng,
  • Shang Lifeng,
  • Jiang Xin,
  • Qun Liu

摘要

Extractive summarization is a widely applied technique in information extraction, information retrieval and natural language processing (NLP) applications. However, traditional methods often struggle to maintain inter-sentence coherence in the extracted summaries. This paper proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and a coherence discriminator module. The coherence discriminator is trained online using augmented sentence sequences. Its task is to judge whether the input sentences (embeddings) form a coherent sequence. Meanwhile, we maximize the coherent scores from the coherence discriminator by updating the parameters of the summarizer. Updating mechanisms of different modules by three learning objectives is designed for effective training. To make the sentence selection procedure differentiable for training, we apply a Gumbel-softmax function with a straight-through trick. Two representation merging strategies are introduced to ensure that the coherence discriminator always takes a sequence of sentence representations without masking vectors. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in extracted summaries based on their positions in the original article, while goodness in terms of relevance metrics (i.e., Rouge scores and BertScores) are preserved. In addition, model-based evaluation (UniEval) and human evaluation also evidence the significant improvement in coherence and consistency of the extracted summaries given by the enhanced extractive summarizer.