Interpreting Personality Traits in Social Media Images Through Visual Question Answering

This paper explores the intricate dynamics of visual question answering (VQA), particularly focusing on the interplay between image content, question semantics, and biases such as age, gender, and race. With the rapid advancements in artificial intelligence and computer vision, automated systems have gained significant traction in answering questions about images. However, these systems often reflect and perpetuate societal biases, including those related to age, gender, and race. We propose an in-depth investigation into how different aspects of images, including facial expressions, influence the answers generated to questions, especially concerning the subjects’ age, gender, and perceived racial characteristics. Here are three fixed questions and five fixed answers (big five personality traits). Moreover, we extend our analysis to incorporate insights from the Big Five personality traits—openness, conscientiousness, extraversion, agreeableness, and neuroticism—into the question answering process. Understanding how these personality traits influence the interpretation of images and subsequently affect the generated answers adds a layer of complexity to our study. By analyzing a diverse dataset encompassing various demographics, including age, gender, race, and personality traits, we aim to uncover the biases embedded in existing VQA models and develop strategies to mitigate them. This study contributes to understanding the nuances of VQA systems and highlights the importance of addressing biases in automated processes. The findings have implications for improving the fairness and inclusivity of AI technologies in various applications, including psychology, human–computer interaction, and marketing. Further research is suggested to refine the proposed strategies and enhance the accuracy and reliability of VQA systems.