<p>Selecting the optimal hyperparameters of a machine learning algorithm is a crucial step in developing accurate classification or regression model, as it significantly impacts the model’s performance. Therefore, tuned hyperparameters enhance the model’s ability to learn from data, improve its generalization capabilities, and ultimately lead to better predictions. Machine learning models have been applied to classify data into predefined classes, the process of finding the best hyperparameters efficiently while maintaining or improving prediction accuracy remains a significant challenge. This paper introduces a novel model-based approach for hyperparameter selection in machine learning classifiers. Our method is based on formal mathematical and logical models to guide the hyperparameter exploration process, achieving an effective balance between computational efficiency and classifier performance. Our key idea is to model the search as an adaptive and reactive control problem, allowing to rigorously navigates the hyperparameter space. We validate the effectiveness of our method through extensive experiments across diverse datasets and classifiers, comparing its performance against existing methods and techniques.</p>