Compression of Deep Neural Network

This chapter introduces the third and fourth components to implement embedded AI, model compression, and model compilation. Model compression is a method to reduce the size of deep neural networks without changing the network structure. Assuming that the neural network model has been generated, techniques such as pruning, weight sharing, quantization, binary/ternary, Winograd convolution, etc. can be used to “compress” the neural network. Model distillation is also introduced, a method that learns a small student model from a large teacher model but maintains the accuracy of the teacher model. Model compilation is the process of translating AI models into AI chip instructions. There are many optimization methods. This chapter focuses on the compression compilation co-design method. This method can effectively optimize the size and speed of deep neural network models and greatly shorten the adjustment time of the compression process, thereby enabling the deployment of deep neural network models on embedded devices.