Advances in sensor technology have enabled objective emotion prediction, yet existing approaches often rely on subjective input. This study proposes a pure sensor-based framework for predicting emotion changes, eliminating subjective influence. By integrating anomaly detection, we transform extreme emotion fluctuations from noise into valuable predictive features. Using Isolation Forest, we identify rare but significant emotional shifts, ensuring that they are effectively incorporated into the model. Furthermore, lagged features (lag_window = 5) enhance temporal dependencies, while data augmentation ensures balanced learning in extreme cases. Experimental results demonstrate that Stacking achieves the best performance, with a mean squared error (MSE) of 0.31 on the test set and 0.02 for extreme emotion changes, attaining 98.64% accuracy. These findings highlight the framework’s robustness, particularly in leveraging extreme cases for improved emotion prediction.