A Novel Approach for Deep Learning Based Video Classification and Captioning using Keyframe

Video classification has gained a significant importance in recent years because of the enormous rise in internet usage and because of the emergence of deep learning models as a tool to automatically classify videos. In this work, an innovative approach for video classification and captioning is proposed by extracting keyframes from the videos. A transfer learning approach is used to classify the videos. A 99.76% training accuracy is observed on the ISRO dataset and 99.89% training on UCF101 dataset was achieved by using the proposed algorithm. The proposed system also discusses the traditional approach of video classification using the CNN + RNN architecture and shows that algorithm outperformed the eminent approach. Finally, an approach for video captioning using a DenseNet201 + GRU based encoder-decoder model is compared. A METEOR score of 0.29 was achieved on the ISRO dataset and a METEOR score of 0.299 on the MSR-VTT dataset.