Investigating a Semantic Similarity Loss Function for the Parallel Training of Abstractive and Extractive Scientific Document Summarizers

Scientific document summarization focusses on condensing scientific literature, research papers, and technical documents into concise summaries while preserving crucial scientific concepts, findings, and conclusions. In this work, we present a novel loss function that incorporates semantic similarity, and use it in the parallel training of extractive and abstractive summarizers, thereby improving the performance of the individual summarizer units. The new loss function is a union of the summarizer cross-entropy losses and the semantic similarity losses among the generated and reference summaries. To validate the effectiveness of the proposed loss function joint with the parallel training, the experiments use a combination of four recently state-of-the-art extractive summarizers and four recently state-of-the-art abstractive summarizers. Results indicate that for all combinations, the extractive and abstractive summarizers both gain significant performance boosts. It is conjectured that the new semantic similarity-induced cross-entropy loss combined with the parallel training will improve any combination of quality extractive and abstractive summarizers.