<p>Driver fatigue is a major cause of traffic accidents, significantly impairing attention and reaction time. Traditional detection methods typically rely either on visual data or sensor signals. Image-based approaches suffer from lighting variations, while sensor-based methods are prone to noise interference. Here, a multimodal fusion architecture that integrates visual imagery with tactile signals from flexible sensors using porous composites is proposed to detect driver fatigue states. A convolutional neural network extracts features from the images, while sensor signals are encoded through fully connected layers. The extracted representations are then projected into the same dimensional space for concatenated feature fusion. Experimental results show that the proposed multimodal approach improves recognition accuracy by more than 4% compared with single-modality methods and generalizes reliably across varied appearances and extreme conditions. Furthermore, a detailed evaluation and quantification of fatigue levels have been conducted, which contributes to accident prevention and promotes driving safety.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A multimodal framework for fatigue driving detection via feature fusion of vision and tactile information

  • Kunpeng Li,
  • Wei Yue,
  • Dong-Bin Shin,
  • Ke Bi,
  • Dong Zhao,
  • Yunjian Guo,
  • Yang Li,
  • Jong-Chul Lee

摘要

Driver fatigue is a major cause of traffic accidents, significantly impairing attention and reaction time. Traditional detection methods typically rely either on visual data or sensor signals. Image-based approaches suffer from lighting variations, while sensor-based methods are prone to noise interference. Here, a multimodal fusion architecture that integrates visual imagery with tactile signals from flexible sensors using porous composites is proposed to detect driver fatigue states. A convolutional neural network extracts features from the images, while sensor signals are encoded through fully connected layers. The extracted representations are then projected into the same dimensional space for concatenated feature fusion. Experimental results show that the proposed multimodal approach improves recognition accuracy by more than 4% compared with single-modality methods and generalizes reliably across varied appearances and extreme conditions. Furthermore, a detailed evaluation and quantification of fatigue levels have been conducted, which contributes to accident prevention and promotes driving safety.