Transformer-Based Word Association of Marathi Text

Marathi, a regional language of Maharashtra, exhibits both morphological and linguistic opulence. Its influence extends beyond the borders of India, to various countries. The language encounters a dearth of resources, notably in text processing for intricate tasks like Named Entity Recognition, Text Classification, Question Answering, Sentiment Analysis, and Anaphora resolution. Natural Language Processing emerges as a pivotal term when confronted with the need to parse unstructured textual data. The intricacy intensifies when confronted with lengthy Marathi sentences, where discerning the interplay of words becomes a formidable challenge. This challenge, however, is surmounted through the adept utilization of the Transformer an implementation deeply rooted in the realm of deep learning. In conventional Recurrent Neural Network and Convolutional Neural Network models, the sequential processing of sentences on a word-by-word basis proves inadequate in preserving dependencies inherent in the input text. Conversely, the Transformer leverages the Self-Attention mechanism to discern dependencies and relationships embedded within the input sentence adeptly. This research paper delves into the exploration of associating pronoun with noun in the Marathi sentence. Transformer-based pre-trained models, including Bidirectional Encoder Representations from Transformers (BERT) multilingual base models, mBERT, and XLM-RoBERTa, undergo fine-tuning for downstream tasks, thereby enhancing their capability to identify nuanced associations between words.