Advertisement Image Classification Using Deep Learning with BERT: A Novel Approach Exploiting Textual Features

Newspapers are an important source for job postings, admission notices, tender announcements, sales and promotions, and many other advertisements. However, there are no category-wise search capabilities for advertisements in either the printed versions or the online versions of the newspapers. This category-wise advertisement retrieval requires a classification model that can classify advertisement images into different predefined advertisement classes. This research study presents and compares two advertisement image classification models which use deep learning for advertisement analysis, understanding, and classification thereafter. Two datasets including Advertisement Image Dataset and a corresponding Advertisement Text Dataset have been created to enable this research. The first classification model presented in this study is a common approach of using convolutional neural network (CNN)-based image classification model in which a pretrained residual network with 50 layers (ResNet50) model is fine-tuned on visual (image only) features using Advertisement Image Dataset giving 70.8% accuracy. The second classification model is a novel approach for advertisement image classification which exploits the textual features and fine-tunes pretrained bidirectional encoder representations from transformers (BERT) base model on Advertisement Text Dataset giving 96.7% accuracy. This significant improvement of 25.9% implies that exploiting textual features in advertisement image classification provides better image understanding leading to much higher classification accuracy than visual features.