Framework for Embedded Neural Network Applications

This chapter introduces the fifth component to implement embedded AI, the embedded AI application framework. The energy efficiency of running neural networks in embedded devices can also be significantly improved through clever application-level optimizations. This chapter introduces the composition of this hierarchical cascade system, analyzes some key factors that can bring about efficiency improvements, and uses a case to demonstrate the cost efficiency improvements brought by this system. This chapter further extends this framework, distributes it to the cloud and devices, and proposes a third implementation model of embedded artificial intelligence: the device-cloud collaboration mode.