The classification of Gallbladder disease is important for accurate diagnosis and treatment planning. Machine learning and deep learning have notable potential in this classification. For a very long time, traditional classification techniques faced hurdles like data imbalance, poor generalization, and a lack of coherence. To complement these shortcomings, this paper presents a two-step approach combining deep learning and ensemble methods together with sophisticated architectural designs and Grad-CAM for broader model understanding. This work employed a dataset that consisted of 10,692 ultrasound images coming from 1,782 patients and used balanced training and data augmentation techniques to tackle data imbalance. The hybrid model outperformed other architectures by achieving an accuracy of 99.02% during testing and also recording the minimum validation loss. While these quantitative results speak of the strength of this model, the real distinguishing aspect of this work is the combination of clinical and imaging data to improve diagnosis accuracy of gallbladder diseases like true and pseudopolyps. These results shed light on the mark that the model is destined to create on clinical work for diagnosis, where it could offer fast and interpretable solutions to real world problems. Next, we plan on validating this model against more heterogeneous data sets and investigating its use in real time ultrasound video analysis.