This paper introduces a deep learning-based emotion analysis method utilizing audio and video data, achieving enhanced accuracy in emotion recognition through comprehensive exploration of both modalities. For audio processing, this paper employed a combination of multi-path fusion convolutional neural networks and channel attention mechanisms to extract emotional features from audio data. The attention mechanism emphasizes the significance of key channels, capturing emotion-specific information more precisely. In the context of video analysis, this paper utilizes the R(2+1)D network, which effectively captures temporal relationships, enabling superior extraction of emotional features from video data. To holistically leverage audio and video information, this paper introduces a cross-modal attention mechanism to merge the two types of features. This fusion method aids in integrating information from different modalities, thereby enhancing the accuracy of emotion analysis. By focusing on the correlation between audio and video, this paper successfully elevates the performance of emotion recognition. Experimental results demonstrate significant advancements on a publicly available dataset (RAVDESS), achieving higher accuracy compared to previous methods. The success of this method lies in the comprehensive utilization of audio and video features and the introduction of a cross-modal fusion mechanism. This paper firmly believes that this approach holds significant research and practical value in the field of emotion analysis, providing a new perspective for a more comprehensive and accurate understanding and analysis of emotional information.