Real-Time Detection of Diverse Highway Abnormalities Using a Two-Stage Vision-Language Model Pipeline

Everyday highway operations are threatened by a variety of abnormal situations, ranging from traffic accidents and congestion to road debris and maintenance activities. These events pose significant risks to traffic safety, highlighting the critical need for real-time monitoring and early warning systems. This paper introduces a novel AI algorithm designed to address these challenges with enhanced speed and accuracy. Our approach integrates a two-stage process: an initial rough incident detection using a FastVLM (Fast Vision Language Model), followed by a more precise analysis with a QWEN 2.5VL model. This second stage leverages predefined descriptions of typical abnormal situations to achieve a deeper understanding of the scene. Our research demonstrates that this new algorithm achieves a significant lower time latency and superior detection accuracy when compared to existing commercial monitoring systems. Furthermore, our model can effectively identify a broader range of complex and new situations, including guardrail damage, road maintenance activities, and early-stage fires, significantly expanding the scope of current detection capabilities. This advancement represents a major step forward in proactive highway safety management.