A Railway Service Platform Oriented Multi-modal Emotion Recognition and Early Warning System

This study addresses the critical need for passenger emotion recognition in railway service scenarios and proposes a multimodal emotion recognition system that integrates video and audio data. The system employs YOLOv5 for human detection and scene understanding in videos. It extracts audio features using a CNN-based network. Cross-modal feature fusion is achieved through the MISA (Multi-modal Information Sensing and Analysis) model. A three-level alert mechanism (normal/level-1/level-2) is designed. A complete visual analytics platform is developed. It supports video uploading, real-time multimodal analysis, alert visualization, and report generation. Case demonstrations verify the feasibility and practicality of the system in railway service platform environments.