Heuristics-Based Hyperparameter Tuning for Transfer Learning Algorithms

Hyperparameters play a crucial role in controlling the learning process, consequently impacting the model performance significantly. In most machine and deep learning paradigms, these hyperparameters must be set explicitly with limited heuristics making the hyperparameter fine-tuning process computationally expensive. This computational complexity gets further aggravated in the transfer learning paradigm involving complex source domain models. This work uses particle swarm optimization for hyperparameter fine-tuning within the transfer learning framework. Specifically, dictionary learning-based self-taught and zero-shot classifiers are proposed for classifying lung diseases and recognizing novel concepts. The hyperparameters in these classifiers are obtained using the particle swarm optimizer by equating the corresponding loss functions to the optimizer’s fitness function; model parameters are obtained as a corollary to this optimization. The proposed self-taught learner with parameters and hyperparameters obtained by particle swarm optimizer achieves a classification accuracy of up to 95.27% on the Covid-19 Lungs CT Scan dataset. The recognition results with the proposed zero-shot learners are 14.5% and 8.9% in the conventional and generalized settings, respectively. Therefore, these results confirm the usefulness of particle swarm optimization in obtaining optimal model parameters and hyperparameters efficiently.