Research on Prediction of Missing Values Based on Multiple Models

In the process of big data analysis, data integrity and consistency are often very important factors that affect the accuracy of analysis results. Therefore, before starting the analysis process, data cleaning work should be done on the collected data sources to ensure that subsequent analysis will not cause errors due to data anomalies. Therefore, maintaining data integrity is a very important task in data cleaning. One of the reasons for incomplete data is that the collected data contains missing values, which are caused by human negligence, instrument failure and other factors during the data collection process. The common methods for dealing with missing values are as follows: ignore the value group with missing values directly, or use the central tendency measurement (such as mean, median, etc.) of the missing value attribute to fill in the missing values. These methods may cause the loss of the original characteristics of the value group, which may affect the output of subsequent data analysis and application, and lead to incorrect results. To address this problem, this study uses machine learning methods to fill in missing values for a single field. We use data without missing values as training data, divide the data into multiple clusters using K-Means clustering method to capture the hidden associations between data, and build prediction models for each cluster using multiple regression and artificial neural network. For missing values that need to be predicted, we first use KNN algorithm to find the cluster to which the data belongs, and then apply the model of that cluster to calculate the predicted value. The experiment proves that the multi-model filling method proposed in this study is better than the existing filling algorithms in terms of root mean square error.