The safety risk due to intrusion in railways is a huge hazard that threatens the safety of train operations. Existing real-time detection algorithms can quickly and accurately detect intrusion. Since different intrusion scenarios have different levels of threat to train operation safety, drivers need to adopt different risk response strategies to avoid risks. There is a lack of a timely and reliable risk assessment method. Quantitative calculation of intrusion risk for different intrusion situations to help drivers make optimal decisions. This paper presents a real-time railway intrusion detection and multi-factor collaborative risk assessment method. This method consists of two parts: intrusion detection and risk assessment. Firstly, intrusion detection in the whole scene is realized by object detection algorithm and semantic segmentation algorithm. Secondly, this method establishes a risk assessment system that evaluates the risk of intrusion based on two major risk factors: the spatial location of the foreign object in relation to the track area, and the cate-gory of the foreign object. Experiments have proved that the intrusion detection algorithm proposed in this method can quickly and accurately realize the railway intrusion detection. The risk assessment system proposed in this method can scientifically, efficiently and real-time realize timely risk assessment of train operation.