Enhancing Classification with Multimodality: A Hierarchical Approach to Fashion Item Categorization

In the dynamic e-commerce landscape, online sellers face the time-consuming task of classifying and categorizing items into appropriate categories. This process is crucial for ensuring product information accuracy and safeguarding consumer interests. Traditionally, manually creating item images and descriptions has been the preferred method to avoid potential inaccuracies and detrimental consequences that could arise from AI-generated errors. However, the advent of deep learning has opened up new avenues for automatic classification tasks. This paper explores the concept of multimodality, a technique that combines an item’s image and description, to achieve high accuracy in hierarchical item categorization. We present a comprehensive analysis of its effectiveness on fashion items and demonstrate how it significantly enhances automatic categorization compared to other approaches that do not consider multimodality.