Early and accurate diagnosis of skin lesions is critical to improve patient livelihood, particularly in populations with limited access to specialized dermatologic care. This study presents a novel framework that integrates clinical metadata with 3D total body photography (TBP) image data. By leveraging advanced preprocessing, metadata encoding and transfer learning with efficentNet, the MelanoAI Framework is designed to classify skin lesions as benign or malignant. Our framework involves multiple innovative architectures which includes a Fourier-based EfficientNet model, memory-augmented networks (CMMANN) and a hierarchical multi-scale fusion network. Precision, recall, and AUC metrics were used to assess the model’s performance. Findings suggest a strong framework for classifying skin lesions by enhancing the overall prediction through the integration of clinical metadata with image-based features. Fourier model has achieved a good AUC score of 0.948 indicating a strong capacity to separate classify both benign and malign lesions. The use of frequency-domain features, metadata-aware alignment, and memory modules significantly improved model generalization and classification performance. These results underscore MelanoAI’s potential in teledermatology and resource-constrained clinical environments, offering a robust and scalable solution for early skin cancer detection.