Video surveillance has become a major monitoring tool because it can provide rich, intuitive and accurate information. However, with the large-scale construction of video surveillance systems around the world, system redundancy has led to a huge waste of Internet resources and information overload, so it is necessary to introduce new optimization algorithms to improve its efficiency. This paper focuses on the application of convolutional neural network in public security video surveillance model, and explains the process of data preprocessing, feature extraction, pattern behavior recognition and system architecture design in detail. Finally, the results of two sets of simulation experiments are as follows: the average accuracy rate of pedestrian recognition in the daytime environment is 96.1%, and the error range is less than 5%, while the comprehensive average accuracy rate of pedestrian recognition at night is 91.3%, and the comprehensive average detection efficiency is improved by 13.3% after the application of deep learning algorithm. The improved public security video surveillance system proposed in this paper not only improves the identification accuracy and detection efficiency of the model, but also lays a solid foundation for the future development of public security surveillance systems.