Deep Learning-Based Multimodal Sensing Framework for Anti-spoofing Systems

This paper presents a multimodal facial anti-spoofing framework based on a Deep Learning approach using a stereo vision and voice sensing system. This framework is part of a project for an intelligent receptionist robot and its function is to authenticate people by applying a face recognition redundancy process that prevents the identity theft of a person. The proposal consists of a smart screen that has a multimodal sensing system including cameras and audio. The redundancy approach pipeline for face recognition contains a face and voice recognition module and a 3D face verification module to determine whether the person is real. The system initially acquires and processes a stream of images and audio in real-time, using deep neural networks it detects and recognizes the face and voice of a person, then aligns the images from two cameras and obtains the 3D point cloud of the detected human face in the scene, and finally uses this 3D information to verify whether the recognized person is real. The proposed solution is evaluated with different people to validate the effectiveness of the system. The dataset will be available for the research community.