TPBFS: two populations based feature selection method for medical data

The high-dimensional nature of medical data frequently results in suboptimal performance of machine learning models. Applying feature selection before classification is necessary to improve the performance of classifiers. Although evolutionary-based wrapper feature selection methods are acknowledged for their superior performance in exploring optimal feature subsets, they have been demonstrated to carry the risk of overfitting and a potential loss of efficient search capability in the later stages of evolution. To address these issues, we propose a generalized wrapper feature selection method called Two Populations Based Feature Selection (TPBFS), which incorporates dual populations evolving in reverse directions to improve convergence speed. It introduces a probability-based crossover operation to mitigate overfitting and a record list to systematically track and replace optimal individuals, which helps to avoid getting stuck in local optima during later stages of evaluation. The experimental results demonstrate that TPBFS is effective in reducing the dimensionality of various medical datasets while guaranteeing the performance of classifiers.