A Theoretical Evaluation of Distributed Large Language Model Inference on a Raspberry Pi 5 Edge Cluster
摘要
We present a theoretical study of distributed Large Language Model inference on a low-cost Raspberry Pi 5 edge cluster. The proposed design uses pipeline parallelism where the transformer layers are partitioned across nodes and executed by lightweight, containerized microservices. This approach targets interactive token generation under constraints on memory capacity, CPU throughput and network bandwidth. Using a quantized transformer model as a reference workload, we outline an analysis in terms of time-to-first-token (TTFT), steady-state tokens-per-second (TPS) and inter-node communication overhead. The analysis suggests that distribution can mitigate single-node memory limits, while end-to-end responsiveness is often dominated by communication latency and synchronization frequency, highlighting a scalability trade-off in edge LLM deployment.