Beyond chat-GPT: a BERT-AO approach to custom question answering system

This work introduces a novel Custom Question Answering (CQA) model leveraging Adam optimized Bidirectional Encoder Representations from Transformers (BERT-AO). This model tackles the challenge of combining textual and visual information for accurate question answering. The core concept lies in understanding the intricate relationships between words in text and the visual features within images. The proposed CQA model with BERT-AO incorporates separate branches for processing text (BERT-AO) and images (CNN), both utilizing self-attention mechanisms. This enables a deeper understanding of context within both textual and visual data. Training is conducted on a comprehensive dataset encompassing factual, open-ended questions, answers, and images, providing a strong foundation for the model’s learning process. The performance of the proposed CQA Model with BERT-AO attains 20.46%, 36.86%, 43.98%, 19.97%, and 17.604% higher accuracy and 27.42%, 41.56%, 42.78%, 18.72% and 16.15% higher F-Score value compared with existing methods like Enhancing Bridge Inspection Visual Question Answering via Pre-Training with External Image-Text Data (VLP), LOIS: Exploring Instance Semantics for Visual Question Answering (LOIS), Bidirectional Multimodal Attention Networks for Visual Question Answering (BiGAN), A Pairwise Attention Framework with Word and Sentence Embedding for Visual Question Answering (DALNet-WSE) and Enhanced GNN Focused Attention and Hybridized Optimization Approach for Visual Inquiry Resolution (GNN-SA-MRFO) respectively. This highlights its effectiveness in combining textual and visual information for superior question answering capabilities.