Surveillance videos play a crucial role in providing evidences. However, the deletion of even a few frames can significantly impact the interpretation of events, while the deletion can be performed easily using video editing softwares without leaving visual traces. This paper introduces a novel method to detect video frame deletion based on velocity field characteristics. The main idea is to convert long videos into a feature sequence, which appears as a sequence containing outlier values for videos with frame deletions, and as a sequence without outlier values for original videos. The proposed approach employs a siamese network to distinguish frame deletions between adjacent frames. A video can be transformed into a feature sequence through the siamese network, and the resulting sequence is fed into a binary classifier for final classification. Experimental results demonstrate the effectiveness of our proposed approach, even for longer videos (e.g., 2000 frames) with minimal frame deletions (e.g., 1 frame).