Dermatological diseases such as fungal infections, psoriasis, and viral skin conditions require accurate classification for timely treatment. In this work, we propose RCA-MViT: Residual Convolutional Attention Multi-scale Vision Transformer, a novel architecture designed to classify skin diseases from medical images. The ResNet18 backbone is incorporated at the deep spatial feature abstraction level, followed by Cross-Attention Vision Transformers that facilitate multi-scale dependencies among the corresponding feature maps. It integrates residual convolutional layers during efficient feature extraction with regard to cross-attention mechanics for the improvement of global contexts across various scales of an input image patch. This dataset contains 8159 images from seven categories—nail diseases, eczema, and malignant lesion. Therefore, by combining the representation for both local and global feature information results in achieving high accuracy in the classification of different types of skin conditions. This research attained an accuracy of 90.50%, along with strong results in metrics such as specificity, ROC-AUC curve, and more. This study shows the integration of residual learning in multi-scale vision transformers for medical image classification. Experimental results validate the effectiveness of RCA-MViT in classifying various dermatological diseases, with potential implications for improving computer-aided diagnostics.