Deep learning models, particularly convolutional neural networks (CNNs), have achieved state-of-the-art performance in medical image classification. However, their deployment in real-world clinical environments is often constrained by hardware limitations, energy requirements, and the time-intensive nature of model fine-tuning. In this work, we propose a lightweight and energy-aware training strategy that decouples feature extraction from classifier optimization. By precomputing features and adapting batch normalization statistics with a sample-aware thresholding mechanism, we reduce computational overhead without sacrificing accuracy. A redesigned classifier head is trained using a margin-based weighted loss, which emphasizes ambiguous cases without requiring end-to-end backpropagation. Experimental results on two widely used medical imaging datasets, Brain Cancer MRI and BreakHis, demonstrate that our pipeline significantly reduces training time and CO2 emissions while achieving competitive or superior accuracy compared to traditional fine-tuning approaches. This makes our method well-suited for resource-constrained settings or rapid prototyping environments.