Fatigue assessment under complex, non-Gaussian vibration loading remains a major challenge in structural dynamics and durability design. Classical spectral methods, though efficient, are limited to stationary Gaussian stresses and linear damage accumulation according to the Palmgren–Miner elementary (PMe) rule. Real-world excitations, however, often exhibit strong deviations from Gaussianity and structural materials shows more complex stress-life behavior than the conservative PMe rule. Thus, this work presents a data-driven extension of the spectral fatigue framework based on higher-order statistical characterization. Through mode shape scaling of variance, kurtosis, and associated spectral moments, fatigue-relevant response statistics of entire finite-element (FE) models are obtained without excessive time-domain simulations. Machine-learning models are trained to estimate fatigue damage for different accumulation laws, including the elementary, original, and Haibach-modified PM formulations. An endurance-crest ratio links the material endurance limit to the statistical scale of the stress response, enabling consistent treatment of these two-parameter S–N models. Validation on an FE model confirms that the proposed approach reproduces reference results with good accuracy while reducing computation time by three orders of magnitude, demonstrating its potential for fast, scalable fatigue life estimation under realistic vibration loading.