Scholarlink

<p>This study analyzes the impact of key-value (KV) representation compression on explainability in lightweight Transformer-based sentiment analysis. DistilBERT and MiniLM are evaluated on IMDB, SST-2, and TweetEval using a two-stage compression framework. The proposed approach achieves compression ratios of 2.46–3.05 while maintaining high representational similarity (cosine similarity &gt; 0.994). Predictive performance remains stable across all settings, with accuracy variations within 0.02. From an explainability perspective, compression effects vary depending on dataset characteristics. SST-2 demonstrates strong robustness, maintaining explanation preservation above 0.90 with high contrastive consistency (~ 0.89). In contrast, IMDB and TweetEval show moderate reductions in fidelity and stability, although meaningful explanation structures are still preserved (preservation ~ 0.53–0.74). In some cases, fidelity and matching reliability are slightly improved after compression. These results indicate that KV representation compression improves memory efficiency while largely preserving predictive performance and core explanation structures. However, its impact on explainability depends on dataset characteristics such as input length and structural complexity.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Analyzing the impact of KV representation compression on explainability in lightweight transformer-based sentiment analysis

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月24日

查看全文
  • Misun Lee,
  • Yeonghyeon Gu

摘要

This study analyzes the impact of key-value (KV) representation compression on explainability in lightweight Transformer-based sentiment analysis. DistilBERT and MiniLM are evaluated on IMDB, SST-2, and TweetEval using a two-stage compression framework. The proposed approach achieves compression ratios of 2.46–3.05 while maintaining high representational similarity (cosine similarity > 0.994). Predictive performance remains stable across all settings, with accuracy variations within 0.02. From an explainability perspective, compression effects vary depending on dataset characteristics. SST-2 demonstrates strong robustness, maintaining explanation preservation above 0.90 with high contrastive consistency (~ 0.89). In contrast, IMDB and TweetEval show moderate reductions in fidelity and stability, although meaningful explanation structures are still preserved (preservation ~ 0.53–0.74). In some cases, fidelity and matching reliability are slightly improved after compression. These results indicate that KV representation compression improves memory efficiency while largely preserving predictive performance and core explanation structures. However, its impact on explainability depends on dataset characteristics such as input length and structural complexity.