Proactive Auto Scaling Based on Marginal Request Change Analysis for Reducing Tail Latency in Kubernetes Cluster

Auto scaling dynamically adjusts resource allocation based on application metrics to optimize service performance and resource efficiency. In Kubernetes, a state-of-the-art resource management platform, resource scaling is managed using the horizontal pod autoscaler (HPA). However, HPA’s reactive approach is not appropriate for rapidly increasing workloads. Therefore, we propose a proactive horizontal pod autoscaler (p-HPA) to decrease tail latency by proactively allocating resources according to the number of requests. Experimental results show that p-HPA reduces tail latency by 8.2%, 8.8%, and 18.8% across all workloads, compared with HPA.