Unleashing the Potential of Transformers: An Efficient Inference Framework with Split Learning

Although split learning has been widely applied to simpler models such as CNNs, its use in Transformer-based architectures remains underexplored. This study introduces Split-LLM and Split-ViT, two architectures that support efficient computation sharing for Transformer-based inference between mobile devices and cloud servers. By selecting optimal split points inside the models, the proposed method reduces both network congestion and computational load compared with conventional schemes that send raw data directly from the device to the cloud. Experiments show that Split-LLM improves the average inference speed per sample by about 40.5% over non-split baselines, and this advantage grows as the data volume increases. Split-ViT also achieves better performance in image classification. With an appropriate split, the inference time per image is reduced by 6.7% relative to direct cloud offloading. In addition, the method improves GPU load balancing, making better use of edge-side computation while easing the workload on cloud GPUs.