Detecting anomalous behavior in surveillance videos is essential for public safety but remains a challenging task due to the complexity of human actions and the need for real-time detection. Existing approaches often focus on either spatial or temporal features, limiting their ability to provide both contextual understanding and detailed tracking of suspicious elements. To address this gap, we propose a hybrid anomaly detection system that integrates a 3D Convolutional Neural Network (3D CNN) with a YOLO-based object detection model. The 3D CNN was trained to classify sequences of 25 video frames as normal or anomalous by learning spatiotemporal patterns, using 815 videos (400 normal and 415 abnormal) sourced primarily from the UCF-CRIME dataset, as well as RWF2000, publicly available online footage, and the MERL Shopping Dataset. Simultaneously, the YOLO model was trained with frames from the same videos, supplemented with internet images and data from the Weapon Detection Dataset for YOLOv5. It identifies and tracks critical objects within the scene, including individuals, covered faces, and weapons, adding interpretability to the anomaly detection process. The 3D CNN achieved an overall accuracy of 91%, with a precision of 90% for detecting anomalous behavior and 93% for normal behavior. The YOLO model demonstrated precision scores of 76% for abnormal actions, 83% for normal individuals, 77% for covered individuals, and 74% for weapons. These results suggest that combining temporal and spatial analysis significantly enhances detection performance, providing a more comprehensive and interpretable surveillance system. This approach lays the groundwork for future deployment in real-time security environments, particularly in scenarios requiring both anomaly recognition and actionable visual context.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Anomaly Detection System Based on 3D Convolutional Neural Networks and YOLO on Surveillance Videos

  • Valentina Beca,
  • Brayan A. Zamora,
  • Carlos M. Paredes,
  • Simena Dinas,
  • Nicolás Llanos-Neuta

摘要

Detecting anomalous behavior in surveillance videos is essential for public safety but remains a challenging task due to the complexity of human actions and the need for real-time detection. Existing approaches often focus on either spatial or temporal features, limiting their ability to provide both contextual understanding and detailed tracking of suspicious elements. To address this gap, we propose a hybrid anomaly detection system that integrates a 3D Convolutional Neural Network (3D CNN) with a YOLO-based object detection model. The 3D CNN was trained to classify sequences of 25 video frames as normal or anomalous by learning spatiotemporal patterns, using 815 videos (400 normal and 415 abnormal) sourced primarily from the UCF-CRIME dataset, as well as RWF2000, publicly available online footage, and the MERL Shopping Dataset. Simultaneously, the YOLO model was trained with frames from the same videos, supplemented with internet images and data from the Weapon Detection Dataset for YOLOv5. It identifies and tracks critical objects within the scene, including individuals, covered faces, and weapons, adding interpretability to the anomaly detection process. The 3D CNN achieved an overall accuracy of 91%, with a precision of 90% for detecting anomalous behavior and 93% for normal behavior. The YOLO model demonstrated precision scores of 76% for abnormal actions, 83% for normal individuals, 77% for covered individuals, and 74% for weapons. These results suggest that combining temporal and spatial analysis significantly enhances detection performance, providing a more comprehensive and interpretable surveillance system. This approach lays the groundwork for future deployment in real-time security environments, particularly in scenarios requiring both anomaly recognition and actionable visual context.