Bidirectional Legendre memory unit: bidirectional memory for person authentication combining voice and online signature

With the increasing popularity of smart portable electronic gadgets, voice-based online person verification systems have become prevalent. However, these systems are susceptible to attacks where illegitimate individuals exploit the recorded voices of legitimate users, leading to false confirmations—spoofing attacks. To overcome this limitation, this article presents an innovative solution by combining speech and online handwritten signatures to mitigate the risks associated with spoofing attacks in voice-based authentication systems because a person has to be present in front of the system to produce an online handwritten signature. To accomplish this objective, this work proposes a novel bidirectional Legendre memory unit (BLMU), a type of recurrent neural network (RNN), for person authentication (verification) and recognition. The Legendre memory unit (LMU) is an innovative memory cell for RNNs that efficiently retains temporal/non-temporal sequential information over a long period with minimal resources. It achieves information orthogonalization by solving coupled ordinary differential equations (ODEs) and leveraging Legendre polynomials, ensuring effective data representation. The proposed framework for person authentication and recognition comprises seven convolution layers, four BLMU layers, two dense layers, and one output layer. The performance of the proposed BLMU-based deep learning framework has been evaluated on a self-generated/private dataset of combined feature matrix of voice signals and online handwritten signatures in the Devanagari script. To assess performance, experiments have also been conducted using various RNN architectures, such as LSTM, BLSTM, and ordinary differential equation recurrent neural network (ODE-RNN), to have a performance comparison with the proposed BLMU-based deep learning (DL) framework. The results demonstrate the superiority of the proposed BLMU-based DL framework in enhancing the accuracy of person verification systems, making it a promising solution for various real-world applications.