Hidden Markov Models have demonstrated their importance as valuable tools for various time series problems, particularly when context plays a crucial role. The fundamental task of Natural Language Processing (NLP) is Parts-of-Speech tagging, which is an outstanding application. This study evaluates the performance of four decoding algorithms – Viterbi, Posterior Decoding, Beam Search, and Greedy Search across four linguistically diverse languages: English, Hindi, Spanish, and Sanskrit. Standard data sets are used for experimental evaluation, which shows that Posterior Decoding achieves the highest accuracy in English (92.15%) and Hindi (93.38%), while Beam Search performs best in Spanish (91.54%) and Sanskrit (86.17%). The results, measured through precision, recall, F1 score, and accuracy, highlight the adaptability and effectiveness of HMMs in varying morphological complexities. Although the models show strong results overall, limitations such as handling of out-of-vocabulary (OOV) words and scalability are acknowledged. These results highlight the usefulness of HMMs in multilingual NLP and pave the way for further research in efficient decoding strategies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Prediction of Parts-of-Speech Tags Using HMM Model in 4 Different Languages

  • Yashika Arora,
  • Bhumi Sachan,
  • Anshika Goel,
  • Renuka Nagpal

摘要

Hidden Markov Models have demonstrated their importance as valuable tools for various time series problems, particularly when context plays a crucial role. The fundamental task of Natural Language Processing (NLP) is Parts-of-Speech tagging, which is an outstanding application. This study evaluates the performance of four decoding algorithms – Viterbi, Posterior Decoding, Beam Search, and Greedy Search across four linguistically diverse languages: English, Hindi, Spanish, and Sanskrit. Standard data sets are used for experimental evaluation, which shows that Posterior Decoding achieves the highest accuracy in English (92.15%) and Hindi (93.38%), while Beam Search performs best in Spanish (91.54%) and Sanskrit (86.17%). The results, measured through precision, recall, F1 score, and accuracy, highlight the adaptability and effectiveness of HMMs in varying morphological complexities. Although the models show strong results overall, limitations such as handling of out-of-vocabulary (OOV) words and scalability are acknowledged. These results highlight the usefulness of HMMs in multilingual NLP and pave the way for further research in efficient decoding strategies.