Automatic Speech Recognition (ASR) has become a crucial technology in automating transcription and enhancing efficiency in various domains, including tourism-focused call centers. This study investigates fine-tuning of the Whisper ASR model for a domain-specific call center task within the Polish language and tourism sector, addressing challenges related to language-specific complexities, domain adaptation, and background noise. Accordingly, a novel, specialized dataset was developed from Polish call center conversations, capturing authentic customer interactions with various accents, industry-specific terminology, and diverse noise conditions. The Whisper model was fine-tuned on this dataset and evaluated against baseline and commercial systems to assess performance. The results demonstrated that fine-tuning on the custom dataset significantly improved transcription accuracy in domain-specific contexts, while maintaining strong performance on an external benchmark within the same tourism domain. This work highlights the Whisper model’s adaptability and underscores the value of targeted data curation for enhancing ASR in low-resource, domain-specific settings.