MiGRoW: A Multi-perspective Feature Weighting Scheme Using Gini, Entropy, and Mutual Information
摘要
Feature weighting improves model effectiveness by correctly assigning significance to input features. This paper introduces MiGRoW (Mutual Information, Gini, Redundancy and Weighting), a new feature weighting system that combines global and local relevance estimation and redundancy adjustment. Local significance is expressed via Gini impurity in K-Means-based grouped subsets, feature variability is measured in terms of entropy, global significance is measured by mutual information, and redundancy in features is reduced using mutual interaction analysis. By integrating both global and local views, MiGRoW avoids the drawbacks of traditional global only weighting schemes and supports adaptive and context-sensitive importance estimation. The method is tested on four benchmark datasets three from the UCI ML repository and one from Kaggle on a variety of classifiers involving Random Forest, Logistic Regression, Naive Bayes, K-Nearest Neighbors, and Multi-Layer Perceptron. Experimental outcomes show that MiGRoW outperforms state-of-the-art techniques like RAW, LASSO, USP, WB, and TabT consistently across metrics like accuracy, precision, recall, and F1-score. The suggested framework presents a light, interpretable, and efficient solution for feature weighting in high dimensional and heterogeneous data settings.