To address the inadequacies in feature extraction and fusion in Mongolian Speech Emotion Recognition (SER), this paper proposes a Mongolian SER model based on the Whisper (MSER-Whisper). Firstly, a method is introduced to process the layer-wise feature representations extracted from the encoder of the Whisper model. Log-Mel spectrograms are fed into Whisper to obtain comprehensive multi-layer encoder features. After dimensionality reduction, an attention mechanism is constructed over these features to identify the layer-wise weight distributions corresponding to different emotion categories. This approach not only leverages the shared representations learned from large-scale training data by the open-source Whisper model—thus reducing the learning difficulty under low-resource Mongolian scenarios—but also fully exploits the acoustic and speech information captured at different layers of the pre-trained model. In addition, the model employs a cross-attention mechanism, where the final Prosodic Features (PFs) guide the final Spectral Features (SFs) to facilitate correlated emotion mining, enabling dynamic adjustment of weight information for effective feature fusion. Experimental results demonstrate that, compared to various advanced baseline models, the proposed method significantly improves the accuracy of Mongolian SER.