This study presents an advanced transfer learning approach for Kannada language speaker recognition in emotionally rich environments, focusing on text-dependent speech data. Emotional variability introduces significant challenges in maintaining speaker recognition accuracy due to alterations in prosodic and spectral characteristics. To address this, we extract log-Mel spectrograms from speech signals, capturing emotion-sensitive time-frequency features. A modified deep convolutional neural network architecture, VGG-24, extended from the standard VGG-19, is employed to enhance the capacity for complex feature extraction tailored to emotional speech. This modified network is fine-tuned on a curated Kannada emotional speech dataset, where speakers utter consistent phrases across various emotional states (e.g., anger, happiness, sadness). The experimental results demonstrate that our VGG-24-based system achieves superior performance compared to conventional models, particularly under emotional variability. This work underscores the potential of transfer learning with deep architectures in improving speaker recognition accuracy for low-resource languages in emotionally dynamic conditions. In the proposed work, we extracted two types of features—Mel-Frequency Cepstral Coefficients (MFCC) and Log-Mel Spectrograms—and trained separate recognition models for each. When evaluated on neutral speech, the Log-Mel Spectrogram model achieved 99.67 % accuracy, compared to 81.81 % for the MFCC model. We then tested both models across all four non-neutral emotions and found that the Log-Mel approach retained its high accuracy under expressive conditions, whereas MFCC performance dropped significantly. These findings demonstrate that Log-Mel Spectrograms more effectively preserve speaker-specific cues across emotional variations, underscoring the importance of emotion-aware feature design for robust speaker recognition in under-resourced languages.