Optimizing Embedded Neural Network Models

This chapter provides an in-depth discussion on model optimization, a key step in embedded AI application development. This includes training-time optimization, post-training optimization, model conversion, model compilation, runtime optimization, and other sub-steps. The pruning, clustering, quantization, compression, and compilation collaboration technologies introduced in the previous chapters are used. In order to deepen readers’ understanding, TensorRT, a model optimization tool designed specifically for NVIDIA chips, is introduced in detail.