TAE: Topic-aware encoder for large-scale multi-label text classification

Convolutional neural networks, recurrent neural networks, and transformers have excelled in representation learning for large-scale multi-label text classification. However, there have been very few works that have incorporated topic information in the process of encoding textual sequential semantics, partly because the text’s topic needs to be modeled separately. To address this, we introduce the latent topic-aware encoder (TAE), designed for large-scale multi-label text classification. The TAE features two key components: a latent topic attention module that correlates latent topic vectors with word hidden vectors and a topic-fused channel attention module that processes topic-specific text representations to produce a refined final text representation. Our experiments demonstrate that TAE seamlessly integrates with existing deep models, significantly enhancing their classification accuracy and convergence speed across various datasets.