Text summarization plays a crucial role in distilling relevant information from large textual documents. However, there are negligible language models available for working on texts available in local or indigenous languages. Regional languages often suffer from a scarcity of digital content, resource limitations, inflections in the nature of the language and variation in the structure of the language. This paper tries to bridge the same gaps allowing us to model regional language text summarizers. In this study, we explore the effectiveness of self-attention mechanisms, specifically leveraging a Conventional Transformer, for abstractive text summarization in two Indo-Aryan languages: Hindi and Gujarati. Our approach involves training a custom-built Transformer model on the ILSUM dataset. The ILSUM (Indian Language Summarization) consists of corpora curated using articles, headlines and summaries from several leading newspapers of the country. Our study, thus, proposes how Transformers are well-suited for text summarization across languages due to their self-attention mechanism, which captures context effectively, their ability to not reply on sequential processing and incorporation of positional encodings.