An explainable deep learning model for dental caries detection and segmentation

Timely diagnosis of dental caries is fundamental to preventive oral care; however, manual interpretation of panoramic radiographs remains labor-intensive and susceptible to diagnostic subjectivity. While deep learning (DL) has demonstrated high performance in medical imaging, its clinical integration is significantly hindered by the "black-box" nature of neural networks and a lack of alignment with clinical risk priorities, such as the high cost of false negatives. To address these limitations, we present a clinical-risk-aware framework for automated caries detection and instance segmentation utilizing the YOLO11-seg architecture. This pipeline enhances generalization under real-world conditions by integrating Bayesian hyperparameter optimization via the Optuna framework with an augmentation-robust strategy tailored for radiographic noise. The model is rigorously evaluated on the COCO-Caries dataset, comprising 2668 tooth-level cropped radiographs. Our optimized YOLO11-seg model demonstrates superior performance over YOLOv8-seg and vanilla YOLO11-seg baselines, achieving a box-level precision of 93.8%, a recall of 75.4%, and an mAP@50 of 85.4%. Critically, these gains are realized while maintaining a rapid inference speed of 5.2 ms and a reduced parameter count of 2.83M, facilitating real-time chairside deployment. By incorporating dual explainability techniques—Grad-CAM for spatial saliency and LIME for model-agnostic local interpretations—the framework provides transparent visual rationales for its predictions. This synthesis of clinical-risk-aware optimization and interpretable artificial intelligence (AI) establishes a robust pipeline for dental diagnostics, effectively bridging the gap between high-performance deep learning and clinical trust.