X-ray images are typically obtained prior to surgical interventions for wrist injuries. With advancements in neural networks, various deep learning models have been widely adopted in computer-assisted diagnosis (CAD) for fracture detection. However, most recent studies focus primarily on model performance, often neglecting model efficiency. This research addresses this issue by proposing a compression pipeline to reduce the computational cost of the detection model. By integrating a compact design with a distillation strategy, our method achieves a reduction of up to 91% in FLOPS and 90% in model parameters, with only a minimal drop of 0.018 in detection accuracy (mAP).