Pantograph-catenary arc poses significant challenges to the dependability and security of high-speed rail systems. Traditional arc detection methods struggle to achieve high accuracy, especially for small-scale arcs in complex backgrounds. This paper proposes MOSSE-YOLOv8, a novel two-stage approach for robust small-target arc detection in pantograph-catenary systems. The first stage employs the Minimum Output Sum of Squared Error (MOSSE) filter to extract the pantograph region of interest, effectively suppressing background interference. The second stage utilizes an improved YOLOv8 model, enhanced with the Small Object Layer (SOL) and the Normalized Wasserstein Distance (NWD) metric, to accurately detect arcs of various shapes and sizes. Extensive experimentation on a real-world high-speed railway dataset prove the superiority of MOSSE-YOLOv8, achieving a provemean Average Precision (mAP) of 94.6% in detecting diverse arc targets under complex scenarios. Compared to baseline models, our approach markedly increases the accuracy, robustness, and efficiency of small-target arc detection in high-speed railway environments. The proposed MOSSE-YOLOv8 model offers promising potential for improving the safety, reliability, and maintenance of pantograph-catenary systems.