Scholarlink

With the incremental use of large language models (LLMs) there is an increasing interest in lightweight substitutes that are easier to train and deploy, specifically in resource-constrained systems. Transformer-based models provide such flexibility, enabling task-specific adaptation with lower computational cost. This research addresses the possible challenges of contextual understanding in underrepresented language, Tamil, by evaluating the effectiveness of primarily 3 transformer model-based approaches: one a custom encoder-decoder model (using RoBERTa and GPT2) and the famous mT5 model. The study uses a synset-based dataset to enhance semantic understanding, followed by fine-tuning with the QA dataset to enhance the contextual understanding. The performance of both models is evaluated using metrics such as METEOR and BERTScore (F1), alongside system-level assessments of training resource utilization, including memory and computational space.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Transformer-Based Models for Tamil QA Tasks: A Comprehensive Study on Transformer Augmentation for Contextual Understanding

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • K. R. Bindu,
  • P. Sanjith Ganesa,
  • Preethi Prabha

摘要

With the incremental use of large language models (LLMs) there is an increasing interest in lightweight substitutes that are easier to train and deploy, specifically in resource-constrained systems. Transformer-based models provide such flexibility, enabling task-specific adaptation with lower computational cost. This research addresses the possible challenges of contextual understanding in underrepresented language, Tamil, by evaluating the effectiveness of primarily 3 transformer model-based approaches: one a custom encoder-decoder model (using RoBERTa and GPT2) and the famous mT5 model. The study uses a synset-based dataset to enhance semantic understanding, followed by fine-tuning with the QA dataset to enhance the contextual understanding. The performance of both models is evaluated using metrics such as METEOR and BERTScore (F1), alongside system-level assessments of training resource utilization, including memory and computational space.