MFLD-RSTF: Multimodal Face Anti-spoofing with rPPG and Deep Spatio-temporal Features

Face anti-spoofing (FAS) is a mechanism to identify and avert presentation attacks in the face recognition systems. The presentation attacks are attempts to deceive the system by presenting a fake face, like as a photo or an image, a video replay, or a 3D face mask, of an authentic user. In this paper, we presented a multimodal face anti-spoofing method (MFLD-RSTF) that combines both texture and rPPG features for FAS. Our architecture is composed of three main modules: (1) A CNNLSTM-based facial feature extractor to extract facial features. (2) A rPPG extractor that computes the rPPG signal from the face region using a frequency analysis technique. (3) A classifier that fuses the facial features and rPPG signal and predicts the spoofing score. We assessed our method on the public datasets: NUAA, SiW, and the last is Replay Attack, and show that it betters existing cutting-edge methods in both intra- and cross-database testing.