Hybrid approach for missing data imputation via correlation-based interpolation and outlier analysis
摘要
This study presents a novel hybrid framework for missing data completion, combining interpolation techniques, variable correlation, and outlier management. In this framework, the target variable with the missing value is determined, and the independent variable with the highest correlation is applied only to these two variables using the Spline and Lagrangian interpolation techniques with the local data window method. Additionally, outliers that may occur due to interpolation are identified using the IQR (Interquartile Range) method and corrected by averaging the non-outlier observations. The experimental study was conducted on the Wisconsin Breast Cancer and Iris datasets, comparing five different imputation methods (Spline, Lagrange, Median, k-Nearest Neighbour (k-NN), and Support Vector Regression (SVR)) under increasing missing-data rates ranging from 10 to 90% for three target variables (“Worst Radius”, “Mean Perimeter”, and “Petal Length”). The findings revealed that spline interpolation provided the highest accuracy in both scenarios, while traditional methods such as median, k-NN, and SVR showed insufficient performance. Lagrange interpolation achieved moderate performance, consistently outperforming the Median, k-NN, and SVR methods, while remaining noticeably less accurate and less robust than Spline interpolation, particularly as the proportion of missing data increased. The proposed framework achieves high imputation accuracy while maintaining computational efficiency, thanks to its correlation-based structure, local interpolation strategy, and outlier correction mechanism.