Vision Systems and IMU Signals to Design a Hand-Free Driving HMI

An experimental analysis of a prototype measurement system to construct a hand-free Human-Machine Interface (HMI) for controlling wheelchairs, integrating face orientation algorithms and Inertial Measurement Units (IMUs) is presented. The aim is to enable individuals with mobility limitations to achieve independent mobility and social interaction. Moreover, the to-be-designed HMI can be an alternative control interface for people who cannot use typical devices. The HMI system utilizes cameras and IMUs to capture the user’s head and torso movements. Experimental results are discussed in comparison of IMU and camera data with the Motion Capture system on accuracy and ease of instrumentation, and it has been demonstrated not only the essential reliability of IMU but also the need for fusion algorithms to enhance camera data accuracy. Thanks to the fusion processing of left and right camera images beside a rider’s face, face orientation detection becomes robust while requesting no inconvenience for the rider such as wearing specially-made mounting devices, etc. The findings suggest a promising approach for developing inclusive and efficient wheelchair control systems.