Speech emotion recognition (SER) plays a pivotal role in human-machine interaction by aiming to accurately identify a speaker’s emotional state through the analysis of speech signals. While Transformer-based models and other large language models (LLMs) have shown progress in this domain, their performance in real-world settings remains inadequate. This is primarily due to challenges in accurately extracting emotional features under noisy conditions and difficulties in capturing key emotional moments. To address these issues, we propose a novel architecture, the Emotion in Real-Scene Transformer (ERT), designed to better capture temporal information by leveraging sequence arrangement. By applying self-attention mechanisms within dynamic windows to identify key emotional segments and facilitating global information exchange across windows, the ERT model effectively enhances emotion recognition in noisy and dynamic real-world environments. Experimental evaluations on the IEMOCAP and MELD datasets show that the ERT model achieves a weighted accuracy (WA) of 72.82% and an unweighted accuracy (UA) of 74.82% on the IEMOCAP dataset, and 53.00% (WA) and 29.30% (UA) on the MELD dataset. The results demonstrate that the ERT model significantly improves emotion recognition by better capturing fine-grained emotional features and handling real-world noise and variability compared to existing advanced methods.