Patching LLMs Efficiently for Edge Devices
摘要
Over-the-air (OTA) updates are essential for maintaining deployed large language models (LLMs) on edge devices, a trend accelerated by the success of compact models such as DeepSeek R1. However, we find existing delta encoding algorithms often perform poorly when patching LLMs and other AI models, as their core assumptions do not hold for model data. We propose ResComp, a residual-based differencing algorithm tailored to the structural alignment and low compressibility of model weights. Instead of indexing the old version and scanning the new for scattered matches, ResComp directly computes the residual sequence between aligned models and compresses it using the bzip3 compressor, which achieves better compression ratios and runs faster than traditional high-ratio alternatives. Extensive experiments on popular open-weight LLMs and Stable Diffusion variants show that ResComp significantly outperforms traditional algorithms in patch size, memory use, and differencing speed. An additional Run-Length Encoder (RLE) enhancement further improves patching speed by ~ 30% on a real edge device, making ResComp an efficient and practical choice for industrial model updates.