The growing complexity of DNN models and the widening gap between compute power and memory bandwidth necessitate fusion dataflows to reduce off-chip memory access. However, designing these dataflows across diverse DNNs and DNN accelerator architectures requires a flexible and accurate scheduling framework to explore the vast design space efficiently. Unfortunately, current state-of-the-art (SotA) frameworks struggle to explore these fusion dataflows by mapping limited fusion patterns on less scalable accelerator architectures. In this paper, we present a fusion dataflow framework called FusionFrame. FusionFrame expands the schedule space by characterizing it from three dimensions: operator fusion, loop tiling, and hardware mapping. To capture the extensive schedule space, we design a memory-centric representation to explore various fusion dataflows. Based on this representation, we develop an analytical model that evaluates on-chip data movement and latency for flexible fusion dataflows, enabling hierarchical fusion of operators on DNN accelerators with multi-level memory architectures. This cost model is validated against a taped-out DNN accelerator, Cambricon-Acc. Extensive case studies are then conducted to explore the performance impacts of various schedule space trade-offs, showing that FusionFrame demonstrates a latency improvement of up to 63.9% compared to SotA methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FusionFrame: A Fusion Dataflow Scheduling Framework for DNN Accelerators via Analytical Modeling

  • Liutao Zheng,
  • Huiying Lan,
  • Xiang Liu,
  • Linshan Jiang,
  • Xuehai Zhou

摘要

The growing complexity of DNN models and the widening gap between compute power and memory bandwidth necessitate fusion dataflows to reduce off-chip memory access. However, designing these dataflows across diverse DNNs and DNN accelerator architectures requires a flexible and accurate scheduling framework to explore the vast design space efficiently. Unfortunately, current state-of-the-art (SotA) frameworks struggle to explore these fusion dataflows by mapping limited fusion patterns on less scalable accelerator architectures. In this paper, we present a fusion dataflow framework called FusionFrame. FusionFrame expands the schedule space by characterizing it from three dimensions: operator fusion, loop tiling, and hardware mapping. To capture the extensive schedule space, we design a memory-centric representation to explore various fusion dataflows. Based on this representation, we develop an analytical model that evaluates on-chip data movement and latency for flexible fusion dataflows, enabling hierarchical fusion of operators on DNN accelerators with multi-level memory architectures. This cost model is validated against a taped-out DNN accelerator, Cambricon-Acc. Extensive case studies are then conducted to explore the performance impacts of various schedule space trade-offs, showing that FusionFrame demonstrates a latency improvement of up to 63.9% compared to SotA methods.