<p>Carbon fiber-reinforced plastic (CFRP) has been widely used in aerospace equipment due to its exceptional properties, such as light weight, high-temperature resistance, and corrosion resistance. Ultrasonic vibration-assisted drilling (UVAD), a new advanced precision machining technology for CFRP, specifically rotary ultrasonic machining technology, has shown promising machining effects. However, existing research on its cutting characteristics, material removal mechanism, and surface quality research is still insufficient. Therefore, this study conducted finite element simulations and machining tests of UVAD on CFRP, focusing on analyzing the effects of high-frequency vibration on the stress–strain state and damage during CFRP machining, aiming to reveal the material removal mechanism under UVAD, as well as the impact of process parameter changes on cutting force. The results show that the cutting force data from both the finite element simulation and the experiment exhibit good consistency, which verifies the accuracy of the simulation model. Additionally, UVAD enhances the cutting edge with high-frequency impact cutting ability, effectively facilitating the fracture removal of brittle fibers, significantly inhibiting burr damage at the hole edge, reducing the drilling axial force by 5.9 to 15.4%, and decreasing the hole exit stratification factor by 3.6 to 6.2%.</p>