<p>Feature selection (FS) is a preprocessing technique that diminishes redundant and non-informative features to enhance data classification methods. This technique has gained global significance with the expansion of real-world data, particularly high-dimensional biological datasets. This study introduces a distinctive wrapper-based FS model, built upon the capuchin search algorithm (CapSA). CapSA is a recent swarm intelligence algorithm inspired by the foraging behaviors of Capuchin monkeys. Despite the strengths of the standard CapSA, it has notable limitations that this paper aims to address through purposeful modifications to the algorithm. These modifications include incorporating genetic algorithm operators (crossover and mutation), a dynamic mechanism for determining the number of leaders, and incorporating adaptive inertia weight. The proposed enhanced variant, named CapSA-CM, aims to achieve a more effective balance between the exploration and exploitation phases of the algorithm. The proposed methods are assessed using high-dimensional, low-sample biological datasets. The CapSA-CM approach is validated by comparing its efficacy with basic and hybrid meta-heuristic algorithms. Statistical analysis demonstrates the superiority of the CapSA-CM in terms of feature reduction, accuracy rates, and fitness values compared to the original CapSA and other comparable algorithms.</p>