Analyzing the Impact of Principal Component Analysis on k-Nearest Neighbors and Naive Bayes Classification Algorithms

Principal Component Analysis (PCA) is a well-known dimensionality reduction technique that has been widely used in various machine learning algorithms. This includes kNN and Naive Bayes algorithms which can be time-consuming. The reduction of dimensions can have positive effects on those two algorithms by reducing the number of related types of data and decreasing the data they need to analyze. Here we present detailed findings about how the PCA algorithm affects them both in time efficiency and accuracy. All calculations regarding those values were carried out in Python programming language. The dataset used in research is the Titanic dataset, on which data cleaning and normalization were done. The data in this paper suggests that it is possible to maintain the same level of accuracy with great improvement in time efficiency. For the kNN algorithm reducing the number of dimensions by one resulted in a 31.09% increase in accuracy and for the Naive Bayes algorithm an 18.18% increase while having an imperceptible effect on accuracy.