Abstract <p>In medical research, the scarcity of labeled data and the high cost of expert annotation present a significant challenge for developing robust classification models, particularly in the context of rare diseases or specialized imaging modalities. To overcome this issue, we propose a three-stage few-shot learning framework that integrates meta-learning with pretraining and fine-tuning. First, during the pretraining stage, we pretrain the feature backbone on labeled external data using supervised loss to learn general feature representations. In the meta-training stage, we replace the fully connected layers of the pretrained model with task-specific fully connected layers and fix the feature extraction parameters. We then meta-train the fully connected layers on labeled simulated tasks using an adaptive learning rate and adaptive regularization with Hard-Mining loss, enabling rapid adaptation to new tasks. Finally, during the target task, we fine-tune the model on the target data, adjusting model parameters to align with the task’s feature distribution. We conducted experiments on challenging medical benchmarks BreakHis and ISIC2018 for few-shot classification tasks. Our method achieves superior performance on medical datasets, significantly outperforming related works. Additionally, ablation studies have also been conducted to validate the effectiveness of each module within the model.</p> Graphic abstract <p></p>