AQLB: Lightweight Adaptive Queue-Aware Load Balancer Microservice Chains

Microservice chains often experience end-to-end latency degradation due to queue buildup, which traditional load balancers fail to prevent. Thus, we present a lightweight Adaptive Queue-aware Load Balancer (AQLB) to tackle this challenge. AQLB combines an online queuing model with short-term linear regression to predict per-replica queuing delay using only runtime metrics. Moreover, we also implement a latency-aware scoring function that jointly weighs predicted delay and inter-node communication cost. The experiments on a Kubernetes cluster with the Sock Shop benchmark and Kong gateway show that our proposal reduces average latency by up to 36% and tail latency by 62% compared to the baseline. Moreover, AQLB achieves a significant fairness balancing index while adding only 117 μs decision overhead. Therefore, our proposal demonstrates a proactive, chain-aware load balancing with negligible complexity.