Efficient Neural Networks and Their Acceleration Techniques for Embedded Machine Learning

This chapter focuses on hardware, software, and machine learning co-design techniques for efficient neural networks. These techniques affect performance, cost, and inference accuracy. This chapter introduces basic compression techniques of machine learning models and some specific cases of lightweight neural network architecture for improving the cost and performance efficiency. This chapter also introduces an SoC (System-on-a-Chip) platform and a simple implementation of DNN (Deep Neural Network) accelerator written in C/C++ on the platform. Selected design techniques, such as quantization, loop unrolling, and pipelining, on the DNN accelerator are quantitatively evaluated in terms of accuracy, latency, and hardware resources to see the design trade-offs.