Sentiment Analysis and Offensive Language Identification in Code-Mixed Tamil-English Languages Using Transformer-Based Models

In multilingual countries like India, code-mixed text is common on social platforms. Current NLP tools, primarily trained on monolingual corpora, struggle with downstream tasks in code-mixed languages. Researchers have tried various methods for Sentiment Analysis (SA) and Offensive Language Identification (OLI) in code-mixed Dravidian languages, but none have combined semantic information from the last three hidden layers of multilingual Transformer models (XLM-RoBERTa-Base, MuRIL and DistilmBERT) with a Genetic Algorithm (GA) based ensembling technique and Multi-Task Learning (MTL) framework. This study proposes such an approach for classifying sentiments and offensive language in code-mixed Tamil using the DravidianLangTech-EACL2022 dataset. Results show that using semantic information from multilingual Transformer models improves performance and GA-based ensembling outperforms standard average ensembling. The best model (Genetic ensembling of XLM-R0BERTa-Base, MuRIL and DistilmBERT with cross entropy loss, AdamW optimizer, learning rate of 1e-5 and dropout of 0.4) achieved a weighted F1-score of 58.00% for Sentiment Analysis and 74.00% for Offensive Language Identification in Tamil-English Code-Mixed dataset.