A Novel Ensemble K-Nearest Neighbours Classifier with Attribute Bagging

Classification of supervised learning in machine learning is a challenging task. K-Nearest Neighbours (kNN) is one of the simple classifiers that is commonly used in many real-life pattern recognition applications. The kNN is a lazy learner as it does not build any decision line from historical training data. In this paper, we have proposed a novel ensemble based K-Nearest Neighbours classifier with attribute bagging technique. The proposed method is designed to reduce the time complexity of the traditional kNN classifier, which also ameliorates the classification accuracy. It employs the attribute bagging technique and selects a sorted sub-set of instances from the original training data to apply kNN classifier. We have applied binary search technique to effectively reduce the search space to find the nearest neighbours. The proposed approach reduces the time consumption of kNN classifier from linear to logarithmic making the algorithm feasible for Big Data. We have tested the performance of the proposed algorithm with traditional machine learning classifiers e.g. kNN, Random Forest, AdaBoost on 10 benchmark datasets. The datasets are taken from UCI Machine Learning Repository. The results show that the proposed method outperformed the basic kNN and AdaBoost classifiers, and is compatible with Random Forest classifier.