ConMix: Contrastive Learning with Mixup Augmentation for Dialogue Summarization
摘要
Seq2seq models have achieved remarkable performance on dialogue summarization, but the exposure bias problem still remains. Contrastive learning has been widely adopted to address this issue. However, previous contrastive learning methods have often been less satisfactory and computationally expensive. Furthermore, recent research on dialogue summarization, which mainly focuses on enhancing model performance by incorporating external knowledge, has also struggled with the exposure bias problem, undermining their summarization capabilities. Therefore, in this paper, we propose a novel contrastive learning framework called ConMix, which constructs positive samples using the dropout mechanism of the transformer and generates negative samples through a mixup of dialogue samples. Additionally, We improve the InfoNCE loss by introducing a mixup coefficient( \(\lambda \) ), termed as MixNCE loss, to balance the model’s sensitivity to contrastive sample pairs. Our experiments on SAMSum and DialogSum benchmark datasets demonstrate that ConMix outperforms recent mainstream methods with less computational cost and achieves a remarkable ROUGE-1 score of 54.23 on SAMSum and 47.37 on DialogSum. In addition, we use t-SNE visualization and conduct a series ablation studies to validate the effectiveness of ConMix.