Domain Knowledge Based Multi-CNN Approach for Dynamic and Personalized Video Summarization

In this paper, we present the Multi-CNN approach for dynamic and personalized Video Summarization. The proposed approach is grounded on Cricket Sport domain knowledge to learn complex and domain features. The personalized video summary is based on individual user preferences and is dynamic (dynamic summary). The considerations of individual user preference, domain knowledge, dynamic content, and Cricket sport make the work one of its kind. The proposed Multi-CNN architecture entails two levels, CNN Level-1 and CNN Level-2. We present domain activity-based video segmentation through CNN Level-1 to generate dynamic video segments. The video segments are then forwarded to CNN Level-2, which includes a stacked organization of two models (Umpire detection and umpire pose recognition) to label the video segments. The individual user preference is matched with labeled video segments for key segment identification. We also propose two novel summary evaluation metrics based on individual user reactions. The results indicate the promising performance of the proposed system and provide significant insights for dynamic and personalized video summarization.