Public protection is required for the identification of fights in surveillance systems, but the current techniques mostly rely on visual information, which has drawbacks in low light and when targets are obscured or their actions are unclear. To improve detection performance, the suggested study presents a novel audio-video integration procedure. Our method distinguishes the sounds of combat activities, including shrieks, verbal attacks, and hitting sounds, by combining CNN-Bi LSTM video-based fight detection with an audio transformer. Combining late fusion between audio and video capabilities creates a method that significantly enhances detection reliability. Our system achieves 89.2% precision and an F1 score of 87.8%, demonstrating greater performance over single-mode alternatives. By creating higher standards that result in more intelligent real-time security solutions, the new research platform uses AI capabilities in violence detection.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Combining Audio and Video Analytics in a Multi-modal Method for Fight Detection in Surveillance Systems

  • Raman Goyal,
  • Lalit Verma,
  • Tushar Gupta

摘要

Public protection is required for the identification of fights in surveillance systems, but the current techniques mostly rely on visual information, which has drawbacks in low light and when targets are obscured or their actions are unclear. To improve detection performance, the suggested study presents a novel audio-video integration procedure. Our method distinguishes the sounds of combat activities, including shrieks, verbal attacks, and hitting sounds, by combining CNN-Bi LSTM video-based fight detection with an audio transformer. Combining late fusion between audio and video capabilities creates a method that significantly enhances detection reliability. Our system achieves 89.2% precision and an F1 score of 87.8%, demonstrating greater performance over single-mode alternatives. By creating higher standards that result in more intelligent real-time security solutions, the new research platform uses AI capabilities in violence detection.