Background <p>Current mask-wearing monitoring technologies face bottlenecks in complex scenarios, including high false-negative rates for small targets, ambiguous discrimination under dense occlusion, and poor recognition of “improper wearing.” This study proposes an efficient, robust detection model to enhance supervision and infection control in densely populated areas like hospital lobbies and subway gates.</p> Method <p>A YOLOv11-Star-TripletAttention detection framework was constructed for medical and public health scenarios. Firstly, a lightweight C3k2_Star module was integrated into the backbone to enhance multi-scale feature interaction via Star topology, reducing computational costs while improving sensitivity to distant small faces and mask edges. Secondly, a Triplet Attention mechanism was embedded in the detection head to jointly model channel, height, and width dimensions. This strengthens semantic responses in mouth-nose regions, accurately evaluating coverage integrity and alleviating ambiguity from posture changes, occlusion, and background interference. Experiments were conducted on a self-built multi-scenario dataset covering high-risk areas.</p> Results <p>The experimental results demonstrate that the proposed YOLO11-Star-triple significantly outperforms the baseline YOLO11, achieving a Recall of 0.761 (a 30.0% improvement) and an F1-score of 0.856 (a 17.6% improvement), with an mAP50 of 0.796. Furthermore, the model maintains real-time inference speeds (&gt; 30 FPS) while exhibiting superior robustness in challenging medical scenarios, such as dense crowds in waiting areas, low-light environments, motion blur, and severe occlusion. These capabilities enable the effective identification of high-risk behaviors, specifically the “incorrect wearing” of masks, which are often missed by standard detectors.</p> Conclusion <p>The proposed YOLO11-Star-TripletAttention model effectively addresses the technical challenges of mask-wearing state recognition in complex real-world scenarios, successfully balancing detection accuracy, inference efficiency, and environmental robustness. Mechanism analysis reveals that the C3k2_Star module significantly enhances small target detection, while the Triplet Attention mechanism strengthens the discrimination of intermediate states (e.g., incorrect wearing). Consequently, this study provides an efficient, reliable, and deployable digital solution for personnel flow management in smart hospitals, precise infection prevention and control, and occupational health monitoring, demonstrating significant clinical application prospects.</p>