Video Situation Monitoring Using Continuous Queries

Video situation monitoring is important for many applications such as infrastructure surveillance, traffic monitoring, etc. Currently, situations are monitored either manually using human-in-the-loop or custom algorithms. Manual approach was applicable for short videos. Monitoring situations in hours of long videos manually is difficult and subject to human error. On the other hand, custom algorithms are designed for specific situations and video types. A new algorithm or software package must be written for every new situation type. Both of the above approaches cannot monitor situations automatically. In this paper, we propose an alternative to the above two approaches to facilitate automated situation monitoring by posing situations as queries. The proposed approach minimizes or avoids human involvement and avoids writing new software packages or algorithms for every new situation type. The proposed framework extracts video contents only once using existing video content extraction algorithms. Appropriate data models and new operators and algorithms for efficient situation analysis are required to perform ad-hoc and what if querying on the extracted contents for situation monitoring. This paper extends the traditional relational model with support for representing various extracted content types. The Continuous Query Language (CQL) is also extended with new operators for posing situations as continuous queries. Backward compatibility, ease of use, primitive new operators (including spatial and temporal), and algorithms for efficient execution are discussed in this paper. Finally, query correctness with manual ground truth, efficiency, and the robustness of the algorithms are demonstrated.