IndicBART Alongside Visual Element: Multimodal Summarization in Diverse Indian Languages

In the age of information overflow, the demand for advanced summarization techniques has surged, especially in linguistically diverse regions such as India. This paper introduces an innovative approach to multimodal multilingual summarization that seamlessly unites textual and visual elements. Our research focuses on four prominent Indian languages: Hindi, Bangla, Gujarati, and Marathi, employing abstractive summarization methods to craft coherent and concise summaries. For text summarization, we leverage the capabilities of the pre-trained IndicBART model, known for its exceptional proficiency in comprehending and generating text in Indian languages. We integrate an image summarization component based on the Image Pointer model to tackle multimodal challenges. This component identifies images from the input that enhance and complement the generated summaries, contributing to the overall comprehensiveness of our multimodal summaries. Our proposed methodology attains excellent results, surpassing other text summarization approaches tailored for the specified Indian languages. Furthermore, we enhance the significance of our work by incorporating a user satisfaction evaluation method, thereby providing a robust framework for assessing the quality of summaries. This holistic approach contributes to the advancement of summarization techniques, particularly in diverse Indian languages.