Edge computing for multimedia IoT systems based on FPGA SoCs with integrated CNN accelerators

Convolutional Neural Networks (CNNs) are becoming the most important technique that helps deliver a powerful edge multimedia IoT system (MIoT). Nonetheless, traditional CNNs are characterized by high computation time and high power consumption, which makes them unsuitable for deployment in large-scale MIoT applications. To resolve this issue, we propose a network architecture consisting of edge and centralized cloud computing. The former, which is based on a Field Programmable Gate Array (FPGA), includes an intelligent system based on Artificial Intelligence, Internet of Things (AIoT), and CNN accelerators for the purpose of implementing a Face Recognition (FR) application based on MobileNet. The proposed system achieves a conscious MIoT architecture through an intelligent task-manager implemented on the edge FPGA, which dynamically assigns tasks between the edge and cloud based on image quality. Experimental results demonstrate that the customized hardware accelerator achieves a high throughput of 1160 FPS—a 54% improvement over similar state-of-the-art implementations—while maintaining an ultra-low-power consumption of 2.3W and a recognition accuracy of 99.2%. Concretely, on an identical Pynq Z1 platform and at the same 100 MHz clock, the proposed accelerator raises throughput from the 751 FPS baseline of the most comparable state-of-the-art design to 1160 FPS (a 54% gain), reduces per-frame latency from 1.558 ms to 0.498 ms, and keeps the recognition accuracy within 0.3% of that baseline (99.2% versus 99.5%).