The type of ensemble learning called stacking enhances the classification of patterns by amalgamation of various algorithms. However, traditional stacking models are occasionally faced with the potential of having high confidence levels in inaccurate predictions, hence lowering the reliability especially in sensitive areas such as the medical field. While basic methods such as bagging and boosting solve issues of both bias and variance, they do not contain any means to prevent overconfidence, and applying linear meta-learners reduces the capacity to identify intricate patterns. We present Dynamic MetaStacker a new type of ensemble classifier where decision tree, logistic regression, and support vector machine (SVM) are combined with a neural network as the meta-classifier. This approach introduces a specific loss function which punishes for the overconfident predictions, thus enhancing the quality and resistance. On Cervical Cancer Risk Factors Dataset, Dynamic MetaStacker outperformed all other base learners and traditional stacking modes as follows; improved accuracy by 4. 5%, and recall by 6. 3%. Specifically, the misclassification rates were eliminated and the confidence-penalized loss function also improved reliability. It is well illustrated from the above results that the model is feasible to work with high imbalanced datasets; it can therefore be utilized to provide superior performance for critical applications such as medical diagnosis.