This paper proposes a novel multimodal emotion understanding framework WavXAI-EmoNet, to predict continuous affective states, namely valence, arousal, and dominance (VAD), by fusing speech and physiological parameters. The dataset comprises synchronized audio and galvanic skin response (GSR) signals, including heart rate (HR), SPO2, and infrared (IR) values, collected from 25 students during academic presentations. Acoustic features, specifically the root mean square (RMS) energy and zero-crossing rate (ZCR), are combined with normalized physiological metrics via early feature-level fusion, yielding a compact 5-dimensional feature vector. A knowledge mapping module with rules converts the merged features into continuous VAD labels based on existing affective computing models and is trained separately for every emotional dimension through random forest regressors. For model interpretability, SHapley Additive exPlanations (SHAPs) are utilized to measure feature contributions and identify decision-making insights. The relative weights of MFCCs, RMS, and SpO2 are identified through SHAP analysis, which highlights the performance of the suggested fusion approach. The fusion of speech and GSR features greatly enhances the accuracy in prediction of affective states with an average MSE of 0.0017. The result of the framework is properly determining the emotional intensity of the student-presenter to provide real-time feedback on the emotional state of the student-presenter during presentations. WavXAI-EmoNet provides an explainable and scalable emotion recognition method that is well tailored to education and mental health monitoring applications.