Non-invasive real-time multimodal deception detection using machine learning and parallel computing techniques

Detecting deception reliably has been a sought-after goal for researchers since the early twentieth century. This is due to its high stakes nature, especially when considering the setting under which a deception detection system would be utilized, such as law enforcement sectors, judicial bodies, and criminal investigation bodies. Therefore, recent literature has greatly focused on developing systems with ever increasing accuracies to minimize false positives. Attempts to diversify the sources of data being analyzed and classified as well as the reliance on artificial intelligence technologies have shown great success. But little attention was paid to the applicability of these systems in real-life scenarios. An idle deception detection system needs to exhibit accuracy but also perform in real-time, a feature that is lacking in the current state-of-the-art. In this work, a non-invasive real-time multimodal deception detection system is developed using advanced machine learning techniques. It combines data sources from video and audio streams to extract visual, acoustic, and linguistic features. It also utilizes parallel computing techniques to ensure high performance adequate for real-time usage. Furthermore, a user-friendly graphical user interface was built to facilitate the use of the system. Multiple experiments were conducted to determine its accuracy under various circumstances and combinations of features, the system had a detection accuracy of 99.83% under real-life, high-stakes scenarios using visual and acoustic features, and 91.03% accuracy under controlled environments, while an 89.54% accuracy was achieved using mixed environments and using visual, acoustic, and linguistic features.