<p>In machine learning, feature selection is crucial for reducing computing costs, increasing generalization, reducing dimensionality, and improving model interpretability. Due to multicollinearity and redundancy, traditional approaches often struggle with high-dimensional data. We propose a hybrid framework, fuzzy feature selection using fuzzy C-means clustering and recursive feature elimination (FCM-RFE), that combines fuzzy logic, filter, and wrapper approaches to address these problems. To capture complex relationships, fuzzy C-means clustering first partitions related features into soft clusters. Then, within each cluster, less significant features are repeatedly eliminated using recursive feature elimination with logistic regression (RFE-LR). For precise selection, features are ranked by the strength of their cluster links using a fuzzy membership-based scoring system. RFE can be applied with various base estimators such as logistic regression (LR), random forest, decision tree, or SVM. In our experiments, LR performed best, and alternative estimators were also tested to evaluate the robustness of the framework. Experiments on 18 benchmark datasets using KNN and SVM classifiers evaluated metrics including accuracy, precision, recall, F1-score, specificity, and AUC-ROC. The proposed approach maintained or improved performance while significantly reducing dimensionality, selecting, on average, only 4.1% of the original features. The maximum accuracy was 94.84% for SVM with FCM-RFE and 90.18% for KNN. The proposed method demonstrated effectiveness and scalability for high-dimensional data analysis, outperforming eight state-of-the-art techniques. This framework is suitable for high-dimensional data analysis across various disciplines because it not only improves classification performance but also enhances interpretability and scalability, and reduces the computation cost.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fuzzy feature selection using fuzzy C-means clustering and recursive feature elimination (FCM-RFE)

  • Amit Kumar Saxena,
  • Damodar Patel,
  • Phichsinee Khongja,
  • Phumin Sumalai

摘要

In machine learning, feature selection is crucial for reducing computing costs, increasing generalization, reducing dimensionality, and improving model interpretability. Due to multicollinearity and redundancy, traditional approaches often struggle with high-dimensional data. We propose a hybrid framework, fuzzy feature selection using fuzzy C-means clustering and recursive feature elimination (FCM-RFE), that combines fuzzy logic, filter, and wrapper approaches to address these problems. To capture complex relationships, fuzzy C-means clustering first partitions related features into soft clusters. Then, within each cluster, less significant features are repeatedly eliminated using recursive feature elimination with logistic regression (RFE-LR). For precise selection, features are ranked by the strength of their cluster links using a fuzzy membership-based scoring system. RFE can be applied with various base estimators such as logistic regression (LR), random forest, decision tree, or SVM. In our experiments, LR performed best, and alternative estimators were also tested to evaluate the robustness of the framework. Experiments on 18 benchmark datasets using KNN and SVM classifiers evaluated metrics including accuracy, precision, recall, F1-score, specificity, and AUC-ROC. The proposed approach maintained or improved performance while significantly reducing dimensionality, selecting, on average, only 4.1% of the original features. The maximum accuracy was 94.84% for SVM with FCM-RFE and 90.18% for KNN. The proposed method demonstrated effectiveness and scalability for high-dimensional data analysis, outperforming eight state-of-the-art techniques. This framework is suitable for high-dimensional data analysis across various disciplines because it not only improves classification performance but also enhances interpretability and scalability, and reduces the computation cost.