Natural Language Processing (NLP) for low-resource languages like Swahili remains constrained by the scarcity of annotated datasets, restricting progress in machine translation, sentiment analysis, and speech recognition. Although human annotation is valuable, it is costly, time-consuming, and difficult to scale. This study investigates the use of GPT-4o to generate a Swahili multimodal dataset containing sarcastic, positive, negative, and neutral captions, centered on environmental themes. The dataset was evaluated using LLM-as-a-judge methods, automated metrics, and human validation. Captions exhibited strong grammatical fluency and contextual alignment, with a sentiment classification accuracy of  ~ 79%. Claude and Gemini models rated grammatical correctness and creativity at ~ 98% and ~ 80%, respectively. Lexical diversity was high, especially in sarcastic captions, evidenced by BERTScore F1 (0.8681) and BLEU (0.0172). Human evaluation confirmed coherence and relevance but highlighted difficulties in reflecting culturally grounded sarcasm, such as idiomatic phrasing and regional irony. These findings expose limitations of general-purpose LLMs in modelling sociolinguistic nuance. This study affirms that LLM-generated synthetic data is a scalable and cost-effective alternative to manual annotation. Future work should focus on fine-tuning LLMs with culturally specific corpora to support real-world applications, including content moderation, environmental advocacy, and social media sentiment analysis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Sarcasm Dataset Generation for a Low-Resource Language: Swahili

  • Eugene Kariba Kamau,
  • Noorhan Abbas

摘要

Natural Language Processing (NLP) for low-resource languages like Swahili remains constrained by the scarcity of annotated datasets, restricting progress in machine translation, sentiment analysis, and speech recognition. Although human annotation is valuable, it is costly, time-consuming, and difficult to scale. This study investigates the use of GPT-4o to generate a Swahili multimodal dataset containing sarcastic, positive, negative, and neutral captions, centered on environmental themes. The dataset was evaluated using LLM-as-a-judge methods, automated metrics, and human validation. Captions exhibited strong grammatical fluency and contextual alignment, with a sentiment classification accuracy of  ~ 79%. Claude and Gemini models rated grammatical correctness and creativity at ~ 98% and ~ 80%, respectively. Lexical diversity was high, especially in sarcastic captions, evidenced by BERTScore F1 (0.8681) and BLEU (0.0172). Human evaluation confirmed coherence and relevance but highlighted difficulties in reflecting culturally grounded sarcasm, such as idiomatic phrasing and regional irony. These findings expose limitations of general-purpose LLMs in modelling sociolinguistic nuance. This study affirms that LLM-generated synthetic data is a scalable and cost-effective alternative to manual annotation. Future work should focus on fine-tuning LLMs with culturally specific corpora to support real-world applications, including content moderation, environmental advocacy, and social media sentiment analysis.