Medical imaging plays a crucial role in modern healthcare, serving as a fundamental component for diagnosis and treatment. The selection of imaging modalities for specific diagnostic tasks often involves balancing accessibility, cost, and performance. In this study, we propose a novel approach that utilizes knowledge from a standard modality characterized by higher performance but lower feasibility to enhance the effectiveness of a meta-modality, which is more accessible yet under-performing. Our focus is on applying deep learning techniques in medical image diagnosis by building a lightweight model. This lightweight mapping model utilizes potential representations leveraging the standard modality to improve the training of a model that relies solely on meta-modality. The effectiveness of our approach are illustrated through its use in a clinical setting for multi-task classification of skin lesions, relying on both clinical and dermoscopic images. Our results indicate a notable enhancement within the diagnostic accuracy of the meta-modality, attained without the need for the standard modality during inference. The balanced accuracy increased from 0.87 for clinical images alone to 0.92 for the combined model using both modalities. Furthermore, the experiments were performed repeatedly to ensure their consistency under several random weight configurations.