Scholarlink

<p>In recent years, Transformer-based pre-trained language models have significantly advanced text classification. However, the complementary semantic information embedded in their multilayer hidden representations remains underutilized, often leading to inconsistent performance across different categories and text lengths. To address challenges such as class imbalance, variable text lengths, and ambiguous semantic boundaries, we propose MLF-LAR, a novel Transformer-based text classification model that integrates hierarchical information through a three-stage pipeline: extracting sentence-level representations from multiple layers, modeling inter-layer dependencies via a Graph Attention Network (GAT), and adaptively fusing layer-wise features with a learnable readout mechanism. The adaptive readout dynamically assigns layer weights per sample, regularized by entropy to prevent weight collapse and ensure stable fusion. Evaluated on four public datasets (Ohsumed, R8, MR, and SST-2), MLF-LAR achieves accuracies of 76.16%, 98.13%, 91.39%, and 95.99%, respectively. Ablation studies validate the contribution of each component, while confusion matrix analysis shows that our inter-layer interaction and adaptive fusion mechanisms enhance discriminability among confusable categories, improving overall classification performance and robustness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-Layer feature fusion with Layer-GAT and adaptive readout for transformer text classification

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月16日

查看全文
  • Xinrong Zhan,
  • Xintao Zhan,
  • Junwen Lu,
  • Shaobo Kang

摘要

In recent years, Transformer-based pre-trained language models have significantly advanced text classification. However, the complementary semantic information embedded in their multilayer hidden representations remains underutilized, often leading to inconsistent performance across different categories and text lengths. To address challenges such as class imbalance, variable text lengths, and ambiguous semantic boundaries, we propose MLF-LAR, a novel Transformer-based text classification model that integrates hierarchical information through a three-stage pipeline: extracting sentence-level representations from multiple layers, modeling inter-layer dependencies via a Graph Attention Network (GAT), and adaptively fusing layer-wise features with a learnable readout mechanism. The adaptive readout dynamically assigns layer weights per sample, regularized by entropy to prevent weight collapse and ensure stable fusion. Evaluated on four public datasets (Ohsumed, R8, MR, and SST-2), MLF-LAR achieves accuracies of 76.16%, 98.13%, 91.39%, and 95.99%, respectively. Ablation studies validate the contribution of each component, while confusion matrix analysis shows that our inter-layer interaction and adaptive fusion mechanisms enhance discriminability among confusable categories, improving overall classification performance and robustness.