Cantonese, the native language of the Guangfu people, plays a vital role in the Guangfu cultural identity, with over 120 million native speakers in China. Despite its significance, a significant portion of the Chinese population cannot communicate in Cantonese, posing a barrier to disseminating its culture. With the widespread adoption of mobile devices and smartphones, collecting voice data has become increasingly feasible, alongside a growing demand for intelligent human-computer interaction. However, the intricate pronunciation, tonal variations, and diverse dialects of Cantonese present substantial challenges for its intelligent transcription and translation, crucial for enhancing human-computer interaction. In our study, we leveraged Cantonese dialect speech data to build a model using the Mel Spectrogram method and the OpenAI Whisper model to explore dialect transcription in diverse sophisticated scenarios. We found that this will provide people with more intelligent voice interaction services. The result shows that the base and small versions of the Whisper model can achieve an error rate of 17.52% and 13.07%, respectively. The larger the number of parameters of the whisper model, the better the model performance is, and it will not fall into overfitting too early. The number of parameters has little effect on the model training time. In addition, the whisper model can better adapt to the Cantonese dialect habits. Fine-tuning can reduce the impact of timbre on transcription quality and improve model performance. However, the model’s performance is lacking for poems, idioms, and mixed foreign languages.