Medical data, by its nature, has the wicked problem of class imbalance which affects both model performance and the quality of synthetic data. Traditional oversampling methods typically overlook the internal distribution of the minority class, leading to overrepresentation in dense regions and underrepresentation in sparse ones. This uneven intra-class distribution driven by feature similarity and the presence of both sparse and dense regions can cause models to misclassify minority instances, particularly those located in less populated subspaces. We propose KG-CTGAN, a novel hybrid oversampling framework that integrates K-Means clustering optimized via Glowworm Swarm Optimization with Conditional Tabular Generative Adversarial Network (CTGAN) to address both inter-class and intra-class imbalance in tabular datasets. By improving centroid initialization with GSO, the clustering operation better identifies the internal structure of the minority class, particularly in sparse and borderline areas. This focused clustering enables CTGAN to concentrate synthetic data generation on the regions where it is most required, enhancing class balance and model performance. Evaluated on real-world medical datasets using Random Forest and XGBoost, KG-CTGAN significantly outperforms baseline imbalance data and K-Means SMOTE, achieving up to especially in F1-score with +29% point gain and AUC improvement reaching 0.97 particularly on the Heart Stroke dataset. To ensure interpretability, SHAP and LIME were applied, providing insight into model decisions and enhancing trust in medical predictions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

KG-CTGAN: A Novel Approach Using Glowworm Swarm Optimized Clustering and CTGAN to Handle Imbalance Medical Data

  • Kaikashan Siddavatam,
  • Subhash Shinde

摘要

Medical data, by its nature, has the wicked problem of class imbalance which affects both model performance and the quality of synthetic data. Traditional oversampling methods typically overlook the internal distribution of the minority class, leading to overrepresentation in dense regions and underrepresentation in sparse ones. This uneven intra-class distribution driven by feature similarity and the presence of both sparse and dense regions can cause models to misclassify minority instances, particularly those located in less populated subspaces. We propose KG-CTGAN, a novel hybrid oversampling framework that integrates K-Means clustering optimized via Glowworm Swarm Optimization with Conditional Tabular Generative Adversarial Network (CTGAN) to address both inter-class and intra-class imbalance in tabular datasets. By improving centroid initialization with GSO, the clustering operation better identifies the internal structure of the minority class, particularly in sparse and borderline areas. This focused clustering enables CTGAN to concentrate synthetic data generation on the regions where it is most required, enhancing class balance and model performance. Evaluated on real-world medical datasets using Random Forest and XGBoost, KG-CTGAN significantly outperforms baseline imbalance data and K-Means SMOTE, achieving up to especially in F1-score with +29% point gain and AUC improvement reaching 0.97 particularly on the Heart Stroke dataset. To ensure interpretability, SHAP and LIME were applied, providing insight into model decisions and enhancing trust in medical predictions.