The proposed study contains the Dual Hesitant Fuzzy set (DHFS) concept extended to a multi-criteria decision making problem for handling the ensemble learning in the feature selection approach. By generalizing the fuzzy, intuitionistic and hesitant fuzzy models, the considered aggregation operator is obtained. Trapezoidal membership and non-membership are defined here. By introducing the aggregation operators for decision making, which include Einstein dual arithmetic, Einstein dual geometric, Einstein dual ordered Arithmetic, and Einstein dual ordered geometric operators, a novel algorithm is proposed, with a numerical example to demonstrate the proposed algorithm. It is contrasted to conventional feature selection methods. The suggested method is also evaluated in comparison to ensemble methods like E-borda, E-wborda, and others. Significance tests are conducted to demonstrate the performance of the suggested algorithm.