To address the issue of bolt detachment and structural failure in the idler shaft support of a specific model of integrated transmission device, finite element modal analysis was employed to determine the first-order natural frequency and mode shape of the support structure. By integrating these results with the gear meshing frequency analysis of the transmission system, it was concluded that the coincidence between the first-order natural frequency of the support structure and the gear meshing frequency induced resonance, leading to the structural failure. A topology optimization approach was applied to enhance the first-order natural frequency of the idler shaft support structure, with the goals of avoiding resonance, reducing vibration amplitudes, and improving overall system performance. A subsequent design iteration and modal analysis were carried out based on the optimization results. The analysis indicated that the optimized support structure’s first-order natural frequency increased from 1756 Hz to 2001 Hz, effectively avoiding the gear meshing excitation frequency of 1755 Hz. To validate the accuracy of the topology optimization and analysis results, bench vibration tests were conducted on two integrated transmission systems—one employing the original idler shaft support structure and the other equipped with the optimized version. The test results demonstrated that the optimized support structure achieved a maximum reduction in vibration energy of 85.7%, confirming the soundness and effectiveness of the optimization scheme derived through finite element-based topology optimization. This methodology offers valuable insights and serves as a reference for the optimization design of structural components in integrated transmission device.