RETSRec: A review-enhanced transformer-based model for sequential recommendation

Sequential recommendation aims to predict the next item a user is likely to engage with based on historical interactions. Traditional methods mainly focus on the order and identifiers of interacted items, often overlooking additional item-related information that could enrich user preference modeling. Many online platforms, however, provide textual reviews that reflect users’ opinions, experiences, and decision-making rationales. Leveraging both sequential behavioral patterns and review content can yield more comprehensive user representations and improve prediction. This study introduces RETSRec (Review-Enhanced Transformer-based Model for Sequential Recommendation), a novel recommendation architecture that integrates sequential behavioral signals with semantic insights derived from reviews. The model employs two encoding branches: 1) a bidirectional transformer to capture dependencies within the ordered interaction sequence, and 2) a BERT-like pre-trained language model followed by a hierarchical attention mechanism to extract high-value semantic features from reviews. By applying multi-head self-attention on aggregated reviews from different items, RETSRec allows modeling contextual dependencies among reviews, thus uncovering consistent behavioral patterns and latent user preferences. Experiments on real-world Amazon datasets demonstrate that RETSRec achieves significant improvements over state-of-the-art methods, underscoring the effectiveness of its design.