Hardware–Software Co-optimization Through Design Space Exploration

Deep learning provides a wide range of neural networks (NNs) with varying accuracy and complexity, which can be mapped to various specialized hardware accelerators. This results in a vast optimization space across energy, latency, area, and task accuracy, complicating the assessment of which NNs, respectively, which hardware architectures, are most efficient as it strongly depends on the hardware platform, respectively, the suite of algorithms to support. Finding the most optimal hardware architecture through exhaustive development of all possible architectural combinations would be extremely slow. The ultimate goal is to rapidly and optimally find the hardware supporting a broad suite of NNs and construct NNs efficiently deployed on a comprehensive set of hardware platforms. This chapter, therefore, brings three contributions: (1) a rapid hardware cost assessment methodology across a suite of networks and architectures using our in-house design space exploration framework, ZigZag, (2) deployment of this analysis for a set of architecture on popular networks for the ImageNet task, and (3) derivation of insights from this study on characteristic of optimal architectures for modern NN models and characteristic of optimal network topologies for modern processor architectures.