Topic modeling is a fundamental task in natural language processing that aims to uncover latent semantic structures from large-scale document corpora. Existing adversarial neural topic models often produce entangled latent representations and suffer from limited interpretability due to the absence of mechanisms that focus on semantically meaningful patterns, thereby impacting coherence and separability of topics while increasing redundancy. To address these issues, we propose an adversarial neural topic model that integrates dual channel attention for enhanced semantic discrimination. Specifically, we incorporate Squeeze-and-Excitation (SE) and Efficient Channel Attention (ECA) blocks within both the encoder and generator to adaptively recalibrate channel-wise feature activations. Our model introduces 1D convolutional channels over TF-IDF inputs to capture local semantic patterns, forming the basis for attention refinement via SE and ECA mechanisms. This promotes better topic separation and coherence by emphasizing salient semantic features. Sparsemax activation is employed in the encoder to yield interpretable and sparse topic distributions without auxiliary regularization. Experimental evaluations on three benchmark datasets show that our attention-guided adversarial model improves topic diversity by up to 0.78, c_v-based topic quality by up to 0.11, and c_a-based topic quality by up to 0.10 over existing adversarial baselines.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Dual-Attention Sparsemax Model for Adversarial Neural Topic Modeling

  • Sookshma Mandala,
  • S. Nagesh Bhattu,
  • Karthick Seshadri

摘要

Topic modeling is a fundamental task in natural language processing that aims to uncover latent semantic structures from large-scale document corpora. Existing adversarial neural topic models often produce entangled latent representations and suffer from limited interpretability due to the absence of mechanisms that focus on semantically meaningful patterns, thereby impacting coherence and separability of topics while increasing redundancy. To address these issues, we propose an adversarial neural topic model that integrates dual channel attention for enhanced semantic discrimination. Specifically, we incorporate Squeeze-and-Excitation (SE) and Efficient Channel Attention (ECA) blocks within both the encoder and generator to adaptively recalibrate channel-wise feature activations. Our model introduces 1D convolutional channels over TF-IDF inputs to capture local semantic patterns, forming the basis for attention refinement via SE and ECA mechanisms. This promotes better topic separation and coherence by emphasizing salient semantic features. Sparsemax activation is employed in the encoder to yield interpretable and sparse topic distributions without auxiliary regularization. Experimental evaluations on three benchmark datasets show that our attention-guided adversarial model improves topic diversity by up to 0.78, c_v-based topic quality by up to 0.11, and c_a-based topic quality by up to 0.10 over existing adversarial baselines.