Aiming at the problems of transmission efficiency and stability reduction arising from double-sided impacts of the spiral bevel gear in the regenerative braking process of monorail vehicles, a double-sided contact impact model of bevel gears is proposed by analyzing the time-varying meshing stiffness (TVMS) and time-varying damping in the process of contact impact. Considering the nonlinear factors such as TVMS, time-varying damping, backlash and transmission error, the double-sided impact dynamic model of monorail vehicle gear transmission system (MVGTS) is established, and the dynamic model of traction motor is coupled for dimensionless processing. The impact characteristics and nonlinear vibration characteristics of gear transmission system during regenerative braking are further studied. The results indicate that during regenerative braking, the spiral bevel gear undergoes multiple alternating impacts between the back-side and drive-side until reaching a complete stop. The MVGTS is susceptible to chaotic motion under significant backlash, while reducing backlash helps alleviate system vibrations and minimize noise. With the increase of damping coefficient, the system will gradually evolve from chaotic motion to periodic motion. Upon reaching a certain damping coefficient, the back-side contact phenomenon will gradually disappear, and a stable drive-side contact state will appear.