GaSUME: A BERT-Covered Genetic Algorithm for Text Summarization

Automatic Text Summarization (ATS) is a fundamental task in natural language processing (NLP), it deals to reduce text size by deleting extraneous data while keeping the original text’s semantic structure. Lately, transformer-based models have achieved remarkable success in ATS and have been regarded as state-of-the-art models for various NLP applications. In this study, we are interested in extractive summarization for a single text, where the purpose is to extract a selection of sentences that best expresses the document’s summary. For automatic text summarization, we propose a combination of Bidirectional Encoder Representations from Transformers (BERT) and a Genetic Algorithm (GA) called GaSUME an improved extension of GaSum system [1], where GA is used as a search space approach and BERT is utilized as a fitness metric. We evaluate our model on the CNN Daily Mail and PubMed available datasets, Our results showed that GaSUME achieves a ROUGE-1 score of 55.89% on the CNN Daily Mail dataset and outperforms the state-of-the-art methods by a significant margin in terms of the rouge score.