Automatic Text Summarization for the Bosnian Language Using LSTM Encoder-Decoder Architecture

Summarization is the process of condensing text into a shorter version while simultaneously preserving its essential information and intended message. While most research to date has focused on the automatic summarization of resource-rich languages, very few studies have been conducted in the South Slavic language group, namely with the Bosnian language. This research focuses on implementing an algorithm for abstractive summarization of texts in the Bosnian language using the encoder-decoder architecture, which handles summarization as a sequence-to-sequence (Seq2Seq) problem. A novel dataset was obtained by crawling the articles and titles of the most relevant Bosnian news websites. The data were processed, cleaned, and lemmatized using a ReLDIanno lemmatizer. This study employs Long Short-Term Memory (LSTM) networks with five stacked bidirectional LSTMs in the encoder phase, a single LSTM decoder, and an attention layer mechanism. The model was trained on a V100 GPU instance provided by Paperspace Gradient. The summaries produced by the trained model were first manually inspected and found to be the most acceptable, with some outlier elements. They were then programmatically evaluated, achieving BLEU (1 to 4) scores of 0.3714, 0.2271, 0.1427, and 0.086, and ROUGE (1, 2, L and W) scores of 16.23, 4.53, 18.74, and 10.58. Finally, the model was compared with existing (albeit scarce) research, and several limitations and suggestions for future work were discussed.