Selecting significant features from the noisy dataset is crucial for every researcher. An ensemble feature selection approach is preferable to enhance the efficacy of the classification approach. Instead of a single filter wrapper method, this study presents a weight rank difference ensemble method followed by a wrapper method with novel binary cat-mouse optimization. In this proposed method, five different filter methods (IG, Relief, GR, mRMR, and SU) are used to select the essential features using the average weight of each one by removing the unstable features. The binary Cat and Mouse-based optimizer explores the search space to determine the optimal features. Combining the ensemble filter-wrapper model induces diversity and produces higher accuracy than the existing methods. Three distinguished classifiers (SVM, NB, DT) are used to evaluate the proposed model. The efficacy of the MWRD-BCMO is examined with 04 different medical datasets. The experimental result demonstrates the impact of the presented method versus existing methods. The MWRD-BCMO method achieves an accuracy of 98.26%, 98.87%, 99.01%, and 99.33% with cervical, colon, Prostrate, and Thyroid datasets, respectively.