In response to the challenges posed by insufficient attention to shot boundary semantics within long-term dependency modeling, a dynamic video summarization algorithm based on contrastive learning and multi-granularity feature fusion (CL-MGFSN) is proposed. First, contrastive learning is introduced to enhance the adaptability of image encoders for video frames; Second, the SVD-KTS algorithm is formulated to segment long and short shots in videos for providing accurate shot boundaries; Finally, the multi-granularity feature fusion summarization network (MGFSN) is designed to capture the semantic associations in frame sequence for enhancing the accuracy of generating summaries. Experiments demonstrate that the proposed algorithm achieves F-score values of 52.3% and 62.4% on the SumMe and TVSum datasets respectively, thus validating its effectiveness in video summarization tasks.