Accurate diagnosis of bone fractures, particularly avulsion and hairline fractures, is essential for effective treatment and recovery. Traditional methods relying on radiologists’ expertise can be subjective and prone to errors. This study investigates the application of feature-based transfer learning with the Xception model to enhance fracture classification in X-ray images. A dataset of 80 hairline fractures, 100 avulsion fractures and 50 non-fracture images were used. Pre-trained Xception convolutional neural network (CNN) models extracted discriminative features, which were then classified using Support Vector Machine (SVM), Logistic Regression (LR), and k-Nearest Neighbors (kNN). The results demonstrated that both SVM and LR achieved high accuracy, with SVM showing superior generalization due to its ability to handle complex, non-linear patterns. LR exhibited reliable performance but faced challenges with non-linear boundaries, while kNN was sensitive to noise and parameter selection. Despite these challenges, the study confirms that feature-based transfer learning improves classification efficiency and accuracy compared to training CNNs from scratch. These findings highlight the potential of integrating deep learning and machine learning for developing automated fracture detection systems to assist healthcare professionals. Future work should explore advanced architectures and refine model parameters to further enhance performance. This study lays a foundation for improving diagnostic accuracy in medical imaging, contributing to better patient care and outcomes.