Irregular Frame Rate Synchronization of Multi-camera Videos for Data-Driven Animal Behavior Detection

Deep learning and camera-based monitoring play a pivotal role in effective farm management. However, reliable data availability remains essential for successful deep-learning applications. Cameras are the primary data sources for computer vision deep learning models. For effective farm management, a multi-camera setup is often used. In a multi-camera farm setup, the input dataset for deep learning is prepared by combining the records of the cameras installed on many sides of the farm. However, an irregular frame rate of various cameras in a multi-camera setup can cause issues such as drift. Therefore, the data from different cameras must be in sync before feeding it to a deep learning model. In this work, we present a method for frame rate synchronization that leverages the timestamp information on the video and achieves high accuracy. Our method addresses a critical use case where the frame rate synchronization is performed post-video recording. Its effectiveness is demonstrated in real-world animal behavior detection scenarios, where precise synchronization is vital. Via this work, we contribute to robust deep-learning models for farm management and livestock analysis by addressing frame rate irregularities.