Word Sense Disambiguation (WSD) is a pivotal challenge in natural language processing (NLP), especially for languages with rich morphological features such as Arabic. In this paper, we introduce a novel approach for Arabic WSD by leveraging the power of stacked ensemble models of BERT-based models. Given the scarcity of sense-annotated resources for Arabic, we first created a new dataset consisting of 51,161 context-gloss pair instances for Arabic verb senses and their context of use, derived from the Arramooz Alwaseet dictionary. Then, we fine-tune four pre-trained models-AraBERT, CAMeLBERT, ArBERTv2, and Arabic ALBERT- on our dataset for the task of binary text classification in WSD. After that, we apply a stacked ensemble technique that combines the strengths of these individual models to achieve superior disambiguation performance. The obtained results demonstrate the superiority of our ensemble strategy over individual baseline models, showcasing its superiority in Arabic WSD. This comprehensive study highlights the importance of advanced ensemble methods in NLP, marking a significant progress in Arabic language understanding and paving the way for further investigations in this crucial area.