V-BERT4Rec: Enhanced sequential recommendation with multi-modal visual information

The goal of this article is to promote sequential recommender systems to a whole new level of understanding and knowledge integration. We introduce a new approach that utilizes the bidirectional encoder representation from transformers to improve recommender systems by incorporating contextual information, such as images representing the items in the recommendation catalog. Our ambition is to demonstrate the importance of combining high-quality research models with knowledge-based information and prove that multi-modal aspects of items can better represent their compatibility with users. More generally, we genuinely quantify the importance of representing the products with more contextual images, sounds, and texts, rather than relying solely on matrix factorization or cross-information of such products. The V-BERT4Rec we introduce in this paper utilizes side unstructured information to improve the results of state-of-the-art sequential recommender systems. Our approach combines the traditional identifier and position embeddings of the BERT4Rec with the newly introduced contextual and visual representations of the potential recommendations. We demonstrate that such an improvement in the sequential recommender system can notably increase performance. The experiments were carried out on a sparse dataset of movie recommendations (MovieLens). The findings have proven that V-BERT4Rec provides a better representation of the movie than BERT4Rec. This is especially true for sparse datasets, where building a genuine representation of the item can significantly help in ranking decisions.