Scholarlink

Deep Neural Network (DNN) training is both compute- and memory-intensive. In this work, we propose a hardware-software co-design approach that leverages ReRAM-based process-in-memory (PIM) technology and second-order training to enhance DNN training efficiency. Second-order training reduces the number of iterations. Importantly, the key operation in second-order training, matrix inversion (INV), can be performed in ReRAM crossbars with \(O\left( 1\right) \) time complexity, minimizing the overhead. However, current ReRAM-based INV circuits face insufficient precision. To overcome this limitation, we propose a high-precision matrix inversion method with 8-bit INV circuit. Building on this foundation, we introduce STAMP, a ReRAM-based PIM accelerator specifically designed for second-order training. Experimental results demonstrate that STAMP achieves an average speedup of 114.8 \(\times \) and an energy saving of 41.9 \(\times \) compared to a GPU counterpart on large-scale DNNs.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

STAMP: Accelerating Second-Order DNN Training Via ReRAM-Based Processing-in-Memory Architecture

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Yilong Zhao,
  • Fangxin Liu,
  • Mingyu Gao,
  • Xiaoyao Liang,
  • Qidong Tang,
  • Chengyang Gu,
  • Tao Yang,
  • Naifeng Jing,
  • Li Jiang

摘要

Deep Neural Network (DNN) training is both compute- and memory-intensive. In this work, we propose a hardware-software co-design approach that leverages ReRAM-based process-in-memory (PIM) technology and second-order training to enhance DNN training efficiency. Second-order training reduces the number of iterations. Importantly, the key operation in second-order training, matrix inversion (INV), can be performed in ReRAM crossbars with \(O\left( 1\right) \) time complexity, minimizing the overhead. However, current ReRAM-based INV circuits face insufficient precision. To overcome this limitation, we propose a high-precision matrix inversion method with 8-bit INV circuit. Building on this foundation, we introduce STAMP, a ReRAM-based PIM accelerator specifically designed for second-order training. Experimental results demonstrate that STAMP achieves an average speedup of 114.8 \(\times \) and an energy saving of 41.9 \(\times \) compared to a GPU counterpart on large-scale DNNs.