As a system integrating intelligent technology and information technology, the intelligent voice learning system plays a very important role in English phonetic teaching. To effectively improve the English speech recognition performance of the intelligent voice learning system, an LPC+CNN-based speech recognition algorithm is proposed. Firstly, English speech signals are acquired by setting a reasonable frame sampling frequency. Secondly, based on Linear Predictive Coding (LPC), the characteristic parameters of the speech signal are obtained, corresponding feature vectors are generated, and input into a Convolutional Neural Network (CNN) for training. Finally, a stable and highly accurate English speech recognition model is obtained by setting a reasonable convolution kernel size and speech recognition accuracy threshold. Simulation results show that this method achieves high accuracy in English speech recognition, with a maximum recognition accuracy of 96.40% and a minimum recognition accuracy of 90.50%.