Skin diseases pose a growing global health concern that is often underestimated and underprioritized in clinical practice. Timely and accurate diagnosis is vital for effective treatment. Recently, Deep Learning approaches, especially Convolutional Neural Networks (CNNs), have shown promise in automating the diagnostic process. However, as the number of disease classes increases, CNNs and traditional Visual Transformers (ViTs) generally struggle with visually similar diseases, resulting in decreased classification performance. To address these challenges, we propose a new approach that combines two advanced ViT architectures—Shifted Window Transformer (Swin) and Multi-Scale Vision Transformer (MViT)—and ensembles these models using a stacking technique to boost classification accuracy. Using a dataset of 19,559 images containing 23 skin disease classes from a Kaggle directory curated from Dermnet, our ensemble achieves 90% accuracy on unseen data, outperforming previous CNN-based approaches, which reached up to 81%. Additionally, the model attains an overall weighted F₁-score of 0.90 and recall of 0.90, demonstrating balanced performance for all classes. Importantly, our proposed ensemble model does not require computationally intensive devices to operate, making it accessible for low-resource clinical environments.