<p>Traditional emotion recognition systems suffer from some problems, such as single-modality dependence, sensitivity to environmental changes, poor real-time performance, and over-reliance on manual feature extraction, which greatly limit their accuracy and robustness. To address the aforementioned problems, this study integrates deep learning with multimodal information fusion methods to enhance the accuracy, real-time capabilities, and robustness of the affective computing interaction system. Facial images and depth information are high-definition cameras and Kinect depth cameras collect and perform image preprocessing is performed to establish a facial expression recognition model based on a convolutional neural network. The AffectNet dataset was used for training and verification. At the same time, voice and text modal data are fused. Multimodal feature fusion is performed using weighted averaging to further enhance the performance of emotion recognition. Finally, an affective computing interaction system is designed and real-time affective state recognition can be achieved, as well as personalized feedback and content recommendations. Experimental results prove that the proposed system is superior to the traditional single-modal systems and support vector machine-based methods with regard to emotion recognition accuracy, real-time responsiveness, stability, and anti-interference ability. With 1,000 pieces of data, the proposed system attained an accuracy of 97.3%, and even at 5,000 pieces of data, an accuracy of 90.6%, and there was no crash or performance degradation during 12 hours of continuous operation.</p>