Identification of Misogynistic Memes Using Transformer Models

The excessive accessibility and the ease of using social media platforms such as Instagram and TikTok have played a significant part in the fleeting escalation of Meme culture, even the Memes that disseminate Misogyny. Hence, in this work, to tackle the problem of online Misogynistic Memes, we explored the usage of the XLM-R Transformer Model in combination with two image Transformer Models, i.e. ViT, and Swin, separately on a benchmarked Meme dataset. We aim to develop a model that can understand the context of the image and text transcribed in it and automatically identify a Meme containing Misogynous content. Hence, the embedding from text and image Transformer Models is concatenated and passed on to single-layered neural network classifier to recognize Misogynistic Memes efficiently. The proposed approach, consisting of two varied Transformer Model combinations, i.e. XLM-R + ViT and XLM-R + Swin, achieves the F1-Score of 0.7468 and 0.7607, respectively.