A New Proposed Algorithm RWH_NN_R to Handle Medical Rare Words in Biomedical Text Document Classification

Computerized digital biomedical text data has been published by medical researchers, doctors, medical professionals, etc., on a daily basis. To manage and find relevant biomedical literature from these vast amounts of publications, the text classification process is helpful to medical professional, researchers in this area. Traditional text classification task directly applying on biomedical text data, rare words are removed which are low frequency medical words. Removing them directly degrades the performance of the classifiers. We proposed a new algorithm for rare word handling using nearest neighbor machine learning concept and text term-class relevancy measure. We have performed comparison of a proposed algorithm with traditional classification methods on different sized four medical cancer datasets. Our experiments show that SVM classifiers outperforms in terms of accuracy and classification time for all datasets using the proposed rare word handling method with little increased total execution time because of rare term identification and handling time in the classification process.