Hydraulic structures subjected to prolonged operational conditions inevitably experience damage and potential structural resonance, which can affect their stability and safety. Structural health monitoring (SHM) is a well-established approach to ensuring safety. However, conventional methods primarily depend on point sensors-insufficient for comprehensive monitoring of large and complex structures. Video-based non-contact monitoring techniques effectively address this limitation. This study investigates the applicability of such methods for monitoring large-scale hydraulic structures in outdoor environments under complex loading conditions. Specifically, a video-based micro-vibration monitoring method utilizing phase-based motion extraction is proposed. First, a video phase-based optical flow method is introduced to extract vibration time histories from any selected pixels or pixel groups in the video, overcoming the time-intensive nature of traditional full-frame image batch processing when capturing large-scale structures with high-speed cameras. Second, the primary factors influencing data accuracy are analyzed by evaluating various shooting angles and scales, with an aging aqueduct serving as the test subject. The results demonstrate that the proposed vibration information extraction method efficiently captures vibration time histories from high-frame-rate videos. The correlation coefficient with sensor data reaches up to 0.96, accurately representing the main frequency bands within the 0–100 Hz range, with particularly strong robustness in the 0–20 Hz range. However, lighting conditions, shooting scales, and angles can affect data accuracy.