Coal mine dispatch telephones serve as crucial communication infrastructure for underground coal mine safety production, playing essential roles in production coordination, emergency response, and accident forensics. Existing monitoring systems suffer from speech information analysis deficiencies. To address this issue, this study proposes an intelligent analysis method based on deep learning for dispatch telephone processing. This work utilize the Whisper pre-trained model for domain adaptation by constructing a coal mine professional named entity library, thereby enhancing the model’s adaptability in coal mine scenarios. Experimental results show that the fine-tuned Whisper model achieves excellent performance in coal mine dispatch telephone speech recognition tasks, reducing character error rate (CER) to 9.7% and significantly improving recognition accuracy for dialects and technical terminology. Subsequently, this study employ generative data augmentation techniques to expand low-category data, combining Bidirectional Encoder Representations from Transformers(BERT) model-based adaptive fine-tuning to alleviate class imbalance issues in safety warning texts, ultimately achieving an F1 score of 85.72% for hazard classification. This research outcome comprehensively enhances coal mine monitoring system construction from the perspective of human voice analysis.