Improving Malayalam Word Sense Disambiguation by Exploiting Various Semantic Features and Vectorization Methods
摘要
A word may give different meanings with different contexts. Such words are called ambiguous words. Word Sense Disambiguation/Detection (WSD) resolves this ambiguity by finding their correct sense. This is an inevitable part of natural language dissemination. This paper proposes various Machine Learning based WSD systems in Malayalam, using algorithms like Support Vector Machines, Naive Bayes, and Random Forest. We justified with results that semantic features like collocations, gloss words, and synonyms of target words could improve the performance of the proposed WSD system. Multiple features like Bag of Words (BOW), word2vec embeddings, and Term Frequency vs. Inverse Document Frequency (TF − IDF) were utilized for feature representation of the input data. Due to the unavailability of a standard dataset, we created three datasets, by selecting three ambiguous words and collecting their corresponding occurrences with different senses, from the web contents. The Support Vector Machine (SVM) classifier with linear kernel, radial basis function (rbf) kernel, and random forest classifiers were observed with the highest accuracy when considered with semantic features.