<p>In the field of medical image classification, the acquisition and annotation of images for training deep models, present significant challenges. Scholars have turned to transfer learning, particularly fine-tuning, as a partial solution to address data scarcity issues. However, when the volume of data in the target domain is insufficient, it leads to redundant parameters in deep models, resulting in overfitting and impacting the ultimate fine-tuning effect. This paper introduces a transfer learning approach based on task-adaptive parameter optimization from the perspective of sparse parameters, primarily applied within medical image classification with Convolutional Neural Networks (CNNs). Initially, individual fine-tuning is conducted on screened convolution kernels from each convolutional layer that are high closely associated with target domain classification, serving as an initial guide for fine-tuning model parameter updates. Subsequently, an adaptive low-rank fine-tuning method is designed, based on the varying contributions of each convolutional layer to classification, less correlated convolution kernels within each layer are adaptive fine-tuned to serve as bias parameters for classification. Experiments with three mainstream CNN models and fifteen medical datasets show that the proposed method can improve the overall fine-tuning efficiency, and the two fine-tuning strategies ensure optimal fine-tuning under the constraint of the number of samples in the target domain.</p>