Graph-Augmented Transformer for Enhanced Structure-Aware Arabic Abstractive Summarization
摘要
Arabic abstractive summarization poses a set of unique challenges as opposed to other languages; Arabic has rich morphology and complex syntax accompanied with scarce high quality annotated resources. Sequential transformer models tend to neglect long range syntactic and discourse relations, hurting overall coherence and coherence with facts. In this paper we propose a novel GA-Transformer: Graph-Augmented Transformer, which explicitly incorporates structural information into summarization. The model creates a graph using dependency parses and coreference chains, presents it utilizing Graph Neural Network (GNN), and effectively integrates structural explicit representations with transformer-based serial encoders through cross-modal attention. This hybrid structure allows the model to utilize local word dependencies as well as global discourse relations when generating summaries. Experiments on AlArabiyaNews and ArXivSumm datasets demonstrate our method achieves the best ROUGE, BLEU, BERTScore results, along with substantial improvement of human-evaluated fluency, coherence and factual accuracy. Ablation studies validate the complementary properties of dependency and coreference graphs in minimizing redundancy, improving coherence and maintaining entity transitions. These results attest to the advantage of integrating explicit linguistic properties in the Transformer-based models for Arabic abstractive summarization.