MotionInsights: Object Tracking in Streaming Video with Apache Flink

MotionInsights facilitates object detection and tracking from multiple video streams in real-time. The system models video processing as a stream processing pipeline. Each video frame is split into smaller blocks, which are dispatched to be processed by a number of Flink operators. Each block undergoes background subtraction and component labeling. The connected components from each frame are merged into objects. In the last stage of the pipeline, all objects from each frame are concentrated to produce the trajectory of each object. The Flink application is deployed as a Kubernetes cluster in the Google Cloud Platform. Experimenting on a 7-machine Flink cluster revealed that MotionInsights achieves up to 6x speedup compared to a non-parallel implementation while providing accurate trajectory patterns. The highest (i.e., up to 6x) speedup was observed with the highest resolution video streams.