Mooring operations are vital for ship safety, where unsafe actions can result in severe accidents. This paper enhances the YOLO v7 network for identifying unsafe actions through video monitoring. The approach includes optimizing anchor points with the k-means++ algorithm, improving the match with unsafe actions, and integrating the Squeeze-and-Excitation (SE) attention mechanism to enhance focus on critical features, boosting recognition performance. Experimental results demonstrate that the improved model outperforms the original YOLO v7, achieving a 0.78% increase in mean Average Precision (mAP) while maintaining the same computational load. This model effectively aids in monitoring unsafe mooring actions. This improved model can serve as a reference for recognizing unsafe action in mooring operations.