Optimization Approaches for Distributed AI Models on Edge Devices
摘要
The exponential growth of Artificial Intelligence (AI) across diverse sectors has notably improved model accuracy but also increased complexity, creating high computational demands. This has led to the centralization of data processing on remote servers with high computing power. However, to ensure data privacy and faster responses, companies are opting for edge devices. These devices, closer to raw data, reduce transmission time and improve response speed, but have less computing power than the cloud. Optimization techniques such as pruning and quantization could be used to simplify complex models on devices by reducing parameters, but this approach might potentially compromise accuracy in some applications. A more effective optimization strategy consists of a distributed implementation, minimizing model changes to preserve precision. In this context, we explore model early exit, which involves the integration of early exits in the model to make preliminary decisions; and model partitioning, which divides complex models into specific sections distributed among different devices. These strategies aim to determine optimal distributions based on available resources and specific application requirements. Both strategies are complementary and can be combined for a more comprehensive approach. In this chapter, we review these optimization techniques, their applications, their challenges and contributions in AI inference.