A Complex Approach to Biometric Authentication Based on Human Face, Speech and Thermal Images
摘要
In this paper, a comprehensive approach to biometric user authentication by combining three independent models and their analyzed features was proposed. The biometric data are supposed to be images of the user’s face, thermal images of the user’s face and user’s speech. An own dataset, consisting of recordings of computer work sessions for 20 users, was used for the study and analysis. Independent authentication models are based on pairwise comparison of data using own neural network regressor trained to calculate the distance between two feature instances. To extract features, we used the FaceNet model in the case of processing face images and thermal images, and MFCCs in the case of processing audio recordings. As feature-level fusion methods were used, namely horizontal concatenation, calculation of median vector values, and calculation of average vector angles. Also considered was the aggregation at the evaluation stage, using the geometric mean, harmonic mean, median, and radar diagram.