Skin cancer is one of the most common cancers worldwide, with melanoma being the subtype associated with the highest mortality rate, even though it is highly curable when detected in its early stages, which makes achieving reliable automated diagnosis especially important. Severe class imbalance, large intra-class variation, and strong visual similarity across cancer categories limit the effectiveness of deep learning models, often causing even advanced architectures to degrade on rare or ambiguous classes. We suggest a unified end-to-end approach that improves sample representativeness, stabilizes inference, and increases feature discriminability in order to overcome these difficulties. Fundamentally, the approach makes use of Deep Metric Learning, which captures the impact of Center Loss in encouraging compact intra-class structure and more distinct inter-class boundaries. We also present Center-Based Down Sampling to choose representative samples close to class centers and minimize redundancy and Test-Time Augmentation to enhance prediction stability via multi-view inference. These components consistently provide complementary improvements, as demonstrated by extensive evaluations across various training configurations, ablation settings, and comparisons with recent state-of-the-art architectures. The complete method achieves 91.16% precision, 88.37% recall, and 89.68% F \(_1\) -score. Overall, more compact embeddings and more accurate predictions across a variety of cancer categories are produced by combining Deep Metric Learning, Center-Based Down Sampling, and Test-Time Augmentation. Our method also achieves competitive performance relative to existing systems such as ConvNeXt-ST-AFF, EFAM-Net, and Enhanced-MobileNet while delivering more reliable per-class behavior.