<p>In unknown road environments, the path chosen by autonomous vehicles significantly impacts suspension vibrations, which directly affects ride smoothness. Active suspension control systems improve vehicle maneuverability and extend the feasible driving range. However, current methods for path planning and suspension control heavily rely on offline models or historical data, limiting adaptability to real-time conditions. To mitigate these limitations, this study utilizes local road surface data captured by onboard optical sensors. These data provide insights into variations in Markov transition probabilities between different environmental states, termed “spatial side information” in this context. By integrating this spatial side information into existing model-based reinforcement learning algorithms, a novel rapid online learning algorithm is developed. This approach enables vehicles to effectively learn and adapt without the need for direct traversal of every state. Furthermore, this research designs optimal control strategies tailored to unfamiliar environments, striking a balance between exploration and exploitation. Safety measures are also integrated to ensure secure exploration by autonomous vehicles. Experimental validation using real-world road data in conjunction with the active suspension test platform confirms the algorithm’s robust performance in learning and control, demonstrating its capability for safe navigation and exploration.</p>