Object detection in video surveillance systems plays an imperative role in numerous applications, including traffic control, human behavior analysis, and security monitoring. With the advancement in deep learning techniques, object detection has made significant progress in videos. This paper offers a focused analysis of the most recent approaches to object recognition in video surveillance using deep learning models, emphasizing their advantages, disadvantages, and possible areas for development. This study analyses one-stage and two-stage modern object identification models and their architectures in detail and evaluates the models' performance using industry-standard datasets and metrics. In addition, we have proposed a taxonomy for object detection models based on traditional, deep learning, and hybrid approaches. The deep learning techniques for object detection include feature-based algorithms, frame differencing, and background subtraction. Subsequently, the review focuses on object detection methods based on deep learning, which revolutionized the field in recent years. The article highlights the fundamental challenges related to object detection in video surveillance, such as occlusion, class imbalance, and illumination conditions. In addition, object detection in several surveillance scenarios—vehicle, pedestrian, and facial recognition—is examined, along with application-specific challenges and future perspectives. It also explores the importance of real-time performance, resource efficiency, and deployment considerations for practical video surveillance systems.