With the rapid shift of the automotive industry towards lightweight design and cost-effectiveness, optimizing vehicle body structures while balancing performance and cost has become a pivotal technical challenge in realizing sustainable automotive manufacturing paradigms. To overcome critical barriers in lightweight technology and address the demand for cost-efficient development, this paper proposes a structural optimization methodology for body-in-white (BIW) design based on a surrogate modeling approach integrated with a lightweight design optimization strategy. Considering the high dimensionality of design variables and the significant computational cost of simulations, a two-stage surrogate modeling framework is developed that effectively integrates physics-based and data-driven approaches. Specifically, the proposed method employs a two-stage approach where local surrogate models first map cross-sectional geometric parameters to sectional characteristics, followed by a deep neural network that predicts overall vehicle performance using equivalent stiffness parameters derived from beam theory. Furthermore, to further enhance the solution efficiency and accuracy of optimization algorithms, this paper proposes a novel CMA-ES-RL hybrid optimization approach that synergistically combines the global search capability of Covariance Matrix Adaptation Evolution Strategy (CMA-ES) with the intensive local exploitation capability of Reinforcement Learning (RL). The developed methodology demonstrates superior performance in BIW structural optimization applications. Experimental results demonstrate that the optimized BIW design achieves an 8% reduction in mass compared to the initial design, while satisfying structural stiffness, strength, and modal performance criteria. Additionally, surrogate model prediction errors remain below 5%, demonstrating that the proposed method effectively meets practical engineering requirements.