Real-time and accurate detection for face and mouth openings in meal-assisting robotics

The detection of facial features and mouth openings plays a major role in field of meal-assisting robotics. Nevertheless, the complexity arises from varying facial orientations, deformations, occlusions, and the diminutive size of mouth regions, making the detection a difficult task. This study proposes an innovative approach, the DFM-YOLO model, tailored for the detection of faces and mouth openings. The YOLOv8s framework was augmented through the incorporation of advanced deformable convolution, aiming to enhance the capacity to grasp spatial details with greater precision. The integration of multi-scale attention mechanisms was employed to boost the responsiveness to diverse input scales. Traditional convolution and C2f components were substituted with more efficient lightweight convolution and aggregation units, respectively, to streamline complexity and minimize floating-point operations. Furthermore, the loss function was refined by integrating EIOU and QFL. The DFM-YOLO framework was benchmarked against the benchmark YOLOv8s model and several state-of-the-art (SOTA) detection models across various datasets. The outcomes indicate that the DFM-YOLO framework excels in terms of accuracy, speed, and robustness. Moreover, we substantiated the efficacy and significance of each enhancement through comparison experiments. The methodology proposed herein can furnish a foundational theoretical guide for the development of intelligent meal-assisting robotic systems.