Efficient Hardware Acceleration of Emerging Neural Networks for Embedded Machine Learning: An Industry Perspective

As neural networks become more complex, the energy required for doing training and inference has resulted in a noticeable shift towards adopting specialized accelerators to meet strict latency and energy constraints that are prevalent in both edge and cloud deployments. These accelerators achieve high performance through parallelism over hundreds of processing elements, and energy efficiency is achieved by reducing data movement and maximizing resource utilization through data reuse. After providing a brief summary of the problems that neural networks have been solving in the domains of Computer Vision, Natural Language Processing, Recommendation Systems and Graph Processing we will discuss how individual layers from each of these different neural networks can be accelerated in an energy-efficient manner. In particular, we focus on design considerations and trade-offs for mapping CNNs, Transformers, and GNNs on AI accelerators that attempt to maximize compute efficiency and minimize energy consumption by reducing the number of access to memory through efficient data reuse.