<p>Missing values are a common feature of real-world datasets, particularly in healthcare data. This can be challenging when applying machine learning algorithms, as most models perform poorly in the presence of incomplete data. The goal of this study is to evaluate the performance of seven imputation techniques: Mean Imputation, Median Imputation, Last Observation Carried Forward (LOCF), K-Nearest Neighbor (KNN) Imputation, Interpolation, MissForest, and Multiple Imputation by Chained Equations (MICE) on three healthcare datasets. Various levels of missing data were introduced—10%, 15%, 20%, and 25%—and the imputation techniques were used to fill in the gaps. The methods were compared using root mean squared error (RMSE) and mean absolute error (MAE). The results indicate that MissForest imputation performed best, followed by MICE. Additionally, we examined whether feature selection should be performed before or after imputation, using recall, precision, F1-score, and accuracy as evaluation metrics. The result suggests that performing imputation before feature selection is better. Since there is limited research on the order of imputation and feature selection, and ongoing debate among researchers, we hope the findings of this study will encourage data scientists and researchers to prioritize imputation before feature selection when working with datasets containing missing values.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A comparative study of imputation techniques for missing values in healthcare diagnostic datasets

  • Luke Oluwaseye Joel,
  • Wesley Doorsamy,
  • Babu Sena Paul

摘要

Missing values are a common feature of real-world datasets, particularly in healthcare data. This can be challenging when applying machine learning algorithms, as most models perform poorly in the presence of incomplete data. The goal of this study is to evaluate the performance of seven imputation techniques: Mean Imputation, Median Imputation, Last Observation Carried Forward (LOCF), K-Nearest Neighbor (KNN) Imputation, Interpolation, MissForest, and Multiple Imputation by Chained Equations (MICE) on three healthcare datasets. Various levels of missing data were introduced—10%, 15%, 20%, and 25%—and the imputation techniques were used to fill in the gaps. The methods were compared using root mean squared error (RMSE) and mean absolute error (MAE). The results indicate that MissForest imputation performed best, followed by MICE. Additionally, we examined whether feature selection should be performed before or after imputation, using recall, precision, F1-score, and accuracy as evaluation metrics. The result suggests that performing imputation before feature selection is better. Since there is limited research on the order of imputation and feature selection, and ongoing debate among researchers, we hope the findings of this study will encourage data scientists and researchers to prioritize imputation before feature selection when working with datasets containing missing values.