<p>Deep learning-based object detection is an important application of deep learning methods in visual understanding, video surveillance, robotics, autonomous driving systems, etc. Its outstanding features are powerful feature learning and representation capabilities compared to traditional object detection methods. To solve the problems proposed, the method, missing object detection in Surveillance Videos Based on Convolutional Gated Recurrent Neural Networks (CGRNN). In short, this research makes tracking applications effective and efficient. First, we obtained a pre-trained model using video frame images as input to Median filters. We generated 64*64 pixel images for missing object detection using the converted video dataset used in the pre-trained model. The pre-processing steps are utilised for reducing the quality of the image. The third step is to extract the colours of video images using the colour map values to estimate that this pixel will be a relative feature weight based on the Relative Scalar Color Histogram (RSCH). The fourth step is Feature extraction is to find out the features and match the image feature from one video frame object based on the maximum threshold weights using Convolutional Neural Network (CNN) with Maximum Standard External Regions (MSER) used for background features extracting. It's applied to improve features quality analysis and Feature support rate. Before classification, devoted to training the features with soft-max logical activation function with Convolutional Gated Recurrent Neural Networks (CGRNN) to classify the missing object from the video using images collected through video type dataset to train the images into the deep neural network. This results in higher classification accuracy, Precision, recall under F-values, and lower error rates. This enables a more achieved classification than other existing methods.</p>