In multilingual cultures, it’s familiar for speakers to switch between two or more languages during conversation—a phenomenon known as code-mixing. Computational models that can efficiently analyse and comprehend code-mixed data are becoming increasingly necessary as code-mixed content becomes more common on social media, messaging apps, and other digital mediums. However, training reliable multilingual natural language processing (NLP) models is complicated by the lack of such code-mixed datasets. We want a framework for creating Code-Mixed datasets to overcome the lack of training data caused by costly acquisition and privacy regulations. This is the first study to provide an excellent overview of works from 2010 to 2023 and offers a thorough analysis of the methods, linguistic concepts, and computational techniques used to generate code-mixed text data. We investigate various strategies, such as rule-based systems, machine learning(ML)models, deep learning(DL) models, and hybrid methods. This assessment attempts to identify existing research gaps and stimulate future research scope for constructing multilingual applications in addition to indicating the state-of-the-art in code-mixed text data and a few standard measures and hybrid evolution metrics for evaluating generated Code-mix data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Building Code-Mixed Datasets for Multilingual Data Analysis: Methods, Metrics and Challenges

  • S. Senthamizhselvi,
  • S. Chitrakala

摘要

In multilingual cultures, it’s familiar for speakers to switch between two or more languages during conversation—a phenomenon known as code-mixing. Computational models that can efficiently analyse and comprehend code-mixed data are becoming increasingly necessary as code-mixed content becomes more common on social media, messaging apps, and other digital mediums. However, training reliable multilingual natural language processing (NLP) models is complicated by the lack of such code-mixed datasets. We want a framework for creating Code-Mixed datasets to overcome the lack of training data caused by costly acquisition and privacy regulations. This is the first study to provide an excellent overview of works from 2010 to 2023 and offers a thorough analysis of the methods, linguistic concepts, and computational techniques used to generate code-mixed text data. We investigate various strategies, such as rule-based systems, machine learning(ML)models, deep learning(DL) models, and hybrid methods. This assessment attempts to identify existing research gaps and stimulate future research scope for constructing multilingual applications in addition to indicating the state-of-the-art in code-mixed text data and a few standard measures and hybrid evolution metrics for evaluating generated Code-mix data.