Traditional sign language communication methods, such as professional interpreters, textual communication, or direct sign language communication, have limitations influenced by the surrounding environment and are unable to fully address the challenges of communication. Therefore, this paper introduces a dynamic sign language recognition algorithm based on convolutional neural networks in complex background. Firstly, create a sign language dataset and perform preprocessing operations, and annotate 21 key points of the hand to improve image quality. Secondly, the processed sign language dataset images are subjected to dynamic sign language segmentation using a combination of three-frame difference method and skin color detection, in order to reduce the impact of image background on recognition and achieve sign language recognition in complex backgrounds. Finally, a new sign language recognition network model is built based on convolutional neural networks to extract key features from sign language images. The model achieves a recognition accuracy of 93.7% on the test set. And based on this model, this study designed a sign language recognition system to assist individuals with aphasia in achieving barrier-free communication.