With the rise of the digital age, the increased use of deep-fake media-synthetic changed photographs, videos, and audio-is threatening information authenticity, privacy of individuals, and public trust. In this study, we propose a multi-modal deepfake detection framework that utilizes machine learning and deep learning techniques to identify manipulated content across three primary domains—images, videos, and audio. The input files are uploaded in the authenticated website. In this model, the deepfake or morphed files will be detected using multiple AI/ML algorithms, for instance, Convolutional Neural Network (CNN) for Images, additionally Long Short-Term Memory (LSTM) for Videos and Mel-Frequency Cepstral Coefficients (MFCC) & Deep Neural Network (DNN) or LSTM for Audios, finally Generative Adversarial Networks (GANs) on the whole. The output display includes detecting whether the input file is morphed or not, its confidence percentage, detecting the regions/ frames/ segments which are morphed by stating the coordinates of the image/ video/ audio file respectively. The deep-fake detection analysis is performed based the factors like differentiation in lightings, skin color, color grading, impossible scenarios for the images, additionally facial expressions & body languages mismatch with the context, frame inconsistencies for the videos, finally anomaly in the voice, tone, content, context and speech patterns of the segments of the audio.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-modal DeepFake Detection of Images, Videos, and Audio Using AI/ML Techniques

  • Shreeyaa Senthilnathan

摘要

With the rise of the digital age, the increased use of deep-fake media-synthetic changed photographs, videos, and audio-is threatening information authenticity, privacy of individuals, and public trust. In this study, we propose a multi-modal deepfake detection framework that utilizes machine learning and deep learning techniques to identify manipulated content across three primary domains—images, videos, and audio. The input files are uploaded in the authenticated website. In this model, the deepfake or morphed files will be detected using multiple AI/ML algorithms, for instance, Convolutional Neural Network (CNN) for Images, additionally Long Short-Term Memory (LSTM) for Videos and Mel-Frequency Cepstral Coefficients (MFCC) & Deep Neural Network (DNN) or LSTM for Audios, finally Generative Adversarial Networks (GANs) on the whole. The output display includes detecting whether the input file is morphed or not, its confidence percentage, detecting the regions/ frames/ segments which are morphed by stating the coordinates of the image/ video/ audio file respectively. The deep-fake detection analysis is performed based the factors like differentiation in lightings, skin color, color grading, impossible scenarios for the images, additionally facial expressions & body languages mismatch with the context, frame inconsistencies for the videos, finally anomaly in the voice, tone, content, context and speech patterns of the segments of the audio.