Efficient Hardware and Software Design for On-device Learning

Deep neural networks (DNNs) are usually trained on servers, and then the trained model is deployed to edge devices. However, the pre-trained models are static and can be inaccurate when the inputs from the new environment are very different from the pre-training data. Therefore, the models on devices need to be continuously adapted by on-device training. To achieve efficient on-device learning, both the software and the hardware issues need to be considered. On the software level, the real-time streaming data usually follows non-independent and identically distribution (non-iid), and simply learning from the latest data can result in forgetting previous data. Besides, the storage on edge devices is usually too small to store all the input data for rehearsal. As for the hardware-level design, implementing on-device training on resource-limited edge devices is challenging because of the complex memory access with different patterns among forward propagation, backward propagation, and weight update. To solve these problems, we first propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Then, we propose an efficient DNN training accelerator, EF-Train, to achieve end-to-end training on resource-constrained, low-power, edge devices.