<p>Stability against perturbations is a defining property of quantum many-body phases of matter. However, most rigorous stabilities are only established for static perturbations; whether any system can remain stable against generic time-dependent perturbations is largely elusive. Here, we identify a universal phenomenon, where the evolving state driven by time-dependent <i>q</i>-local Hamiltonians can be exponentially localized in an energy window of instantaneous spectrum, and prove its survival under generic time-dependent perturbations. Applying such energy-space localization to classical and quantum LDPC codes whose codewords are separated by extensive energy barriers, we show that the system remains localized near the original codeword for an exponentially long time under generic time-dependent perturbations. For classical optimization problems with clustered solution spaces, the stability becomes an obstacle for quantum Hamiltonian-based algorithms to escape local minima. Our work provides a new lens for analyzing quantum non-equilibrium dynamics and tools for establishing stability and designing quantum algorithms.</p>