In this research, Firefly algorithm to generate summary of a given video is proposed. The Firefly Algorithm is an optimization algorithm that draws inspiration from the behaviour of fireflies. There are many types of existing video summarization techniques. Here in this implementation, we focus on summarizing the video based on Object of Interest. Object of interest-based video summarization is a technique for identifying and choosing key portions of a video based on the existence of specified items or regions of interest within the frames. This method involves identifying and tracking relevant things in video frames, and then picking keyframes or short video clips containing those objects or regions of interest. In a surveillance video, for example, the item of interest could be a person or a vehicle entering a restricted area, and the summarization algorithm would select the frames or clips in which the object is present to provide a summary of the video’s relevant events.