As speech recognition technology rapidly advances and becomes widely applied, it has become a key technology in many fields. However, the problems such as information forgetting, context dependence and inaccurate semantic understanding under long speech fragments seriously affect the accuracy and efficiency of speech recognition, and restrict its promotion and application in a wider range of scenarios. In this paper, a speech recognition model based on improved Transformer is proposed, which introduces LSTM network, and pays more attention to the context characteristics of speech data to obtain good recognition results. Finally, the experiment is carried out using AISHELL-3 speech data set, and the experimental results at the end of this paper indicate that the method proposed has low Character Error Rate and Sentence Error Rate. Compared with the other two recognition models, the proposed method reduces the CER and SER by 5.73% and 13.37% respectively compared with Transformer, and reduces the CER and SER by 4.42% and 9.68% respectively compared with DeepSpeech2, verifying the effectiveness of the method.