Speech emotion recognition (SER) plays an important role in human-computer interaction. Recent research focuses on fusion of multiple features extracted from speech. However, the advantages of different features are not well discovered, resulting in limited feature interaction and fusion. In this paper, we propose a two-stage feature fusion method for SER. First, three types of features are extracted from spectrogram, Mel frequency cepstral coefficients (MFCC), and raw speech waveform using time-frequency convolutional neural network (TFCNN), bidirectional long short-term memory network (Bi-LSTM), and pre-trained Wav2Vec 2.0 model, respectively. Then, a cross-feature interactive fusion model is designed, in which the context and time-frequency representations extracted from MFCC and spectrogram are used to guide the Wav2Vec2 features to capture more latent complementary information. Finally, a global fusion model is introduced to enhance the global feature interaction. Experimental results show that the proposed model achieves the state-of-the-art performance on the IEMOCAP and EMODB datasets, with weighted recognition accuracies of 73.28% and 93.89%, respectively.