The vibration of ballast layer is greatly intensified by high-speed train loads compared with those under the normal speeds, which would result in notable settlements of ballasted tracks. Although the macro-response validated discrete element method has been widely used for the dynamic behavior analyses of granular materials, the actual vibration characteristics of ballast particle accompanied with flow behavior in a high-speed railway remain unclear due to the lack of adequate testing validation. Herein, a Particle tracking velocimetry-based monitoring approach was developed upon the formerly established full-scale model testing facility. A ballast layer about 5.5 m in length, 4.5 m in width and 0.35 m in height was prepared, in which Smart Rocks that could record vibration histories were installed. Various actual loading combinations of train speeds up to 360 km/h and axle loads up to 25 tons were applied, during which photos were taken by a camera fixed perpendicularly to the ballast layer. The monitored results indicated that the vibration amplitudes of ballast particles at 15 cm below the sleepers increased by nearly 10 times with the rise of train speed from 100 to 360 km/h. Meanwhile, the vibration amplitude of ballast layer surface at track center under 360 km/h approached 0.82 g in the downward direction, which was close to the weightless state. The transiently disabled interlocking among ballast particles induced significant increases in displacements and rotations of ballast particles and a rapid development of settlement of ballast tracks. The accumulated displacements of surface ballast particles correlated positively with the settlement development of ballast layer. The obtained results disclosed the relationship between particle movements and macro settlement of ballast layer and would provide data support for further DEM-based researches.