When alarming the abnormal state of the construction site, due to the lack of systematicness in the composition analysis of specific construction safety influencing factors at different stages, the accuracy of linkage alarm is relatively high. Therefore, the study on linkage alarm of abnormal state of large pumped storage power station under video monitoring is put forward. Combined with different stages of construction, the existing factors affecting construction safety are comprehensively analyzed from three aspects: construction preparation, foundation and main structure. Based on the image information collected by video surveillance, the existence state of safety influencing factors in the video surveillance picture is identified through the convolution layer including the upper and lower layers, and the operation content that needs to be alarmed at the construction site is determined by combining the correlation between safety influencing factors. In the test results, the design of linkage alarm method not only has high stability for the overall accuracy of alarm in different test conditions, but also keeps above 90.0% all the time.