Lightweight Neural Networks

This chapter introduces the second component to implement embedded AI, lightweight neural networks (abbreviated as lightweight network). Lightweight neural networks have small sizes and can operate within the constraints of low-power and memory-constrained environments while maintaining accuracy. Firstly, several strategies are introduced to reduce the computational complexity of neural networks without sacrificing accuracy. These strategies include the use of grouped convolutions, depthwise convolutions, pointwise convolutions, and depthwise separable convolutions. Next, some classic lightweight neural networks using the above strategies are introduced, such as SqueezeNet, Xception, MobileNet, Mnasnet, etc. By understanding the design ideas of these models, analyzing their network structures, and comparing the performance improvements they bring, we can learn how these models provide the best balance between performance and resource utilization, reducing parameter count and computational overhead. This chapter also lists some other common lightweight neural network models and introduces their application methods.