The rapid progression of Artificial Intelligence has made Vision Transformers (ViTs) crucial for computer vision, offering accuracy comparable to or exceeding that of CNNs. However, ViTs present implementation challenges on hardware platforms, requiring the processing of larger matrices, leading to substantial parameter counts and demanding memory accesses. In response to these challenges, this paper introduces a Configurable Matrix Multiplication Module C3M-ViT. It employs a highly parallel temporal structure based on clusters of Processing Elements (PEs) organized in a two-dimensional matrix. By leveraging a tiling methodology and an optimized dataflow, the C3M-ViT achieves efficient storage without the need for data rearrangement enabling uniform handling of both input and output layers, while maintaining a high PE utilization rate. Furthermore, the C3M-ViT’s weight loading strategy significantly reduces memory footprint by eliminating the necessity to store entire weight matrices. This paper present a 5 \(\times \) 4 cluster matrix configuration with 1,446 parallel PEs, delivering 77.6 GOPs/W energy efficiency, which is approximately three times higher than comparable state-of-the-art accelerators under similar conditions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

C3M-ViT: A Configurable Matrix Multiplication Module for Energy-Efficient Vision Transformer Accelerator

  • Thomas Jacquemin,
  • Hana Krichene,
  • Benoit Tain

摘要

The rapid progression of Artificial Intelligence has made Vision Transformers (ViTs) crucial for computer vision, offering accuracy comparable to or exceeding that of CNNs. However, ViTs present implementation challenges on hardware platforms, requiring the processing of larger matrices, leading to substantial parameter counts and demanding memory accesses. In response to these challenges, this paper introduces a Configurable Matrix Multiplication Module C3M-ViT. It employs a highly parallel temporal structure based on clusters of Processing Elements (PEs) organized in a two-dimensional matrix. By leveraging a tiling methodology and an optimized dataflow, the C3M-ViT achieves efficient storage without the need for data rearrangement enabling uniform handling of both input and output layers, while maintaining a high PE utilization rate. Furthermore, the C3M-ViT’s weight loading strategy significantly reduces memory footprint by eliminating the necessity to store entire weight matrices. This paper present a 5 \(\times \) 4 cluster matrix configuration with 1,446 parallel PEs, delivering 77.6 GOPs/W energy efficiency, which is approximately three times higher than comparable state-of-the-art accelerators under similar conditions.