ServScale: Concurrency-Aware Serverless Execution and Scaling Paradigm
摘要
Many modern cloud applications require intra-instance concurrency to efficiently handle multiple requests or tasks in parallel. While platforms like Kubernetes are widely adopted for scalable deployment on Infrastructure-as-a-Service (IaaS), this approach often leads to resource inefficiencies due to data center tax and idle containers. Serverless computing provides elastic scaling and pay-as-you-go benefits, but migrating such applications remains challenging. Most function containers of existing serverless platforms process requests serially and rely solely on horizontal scaling, forcing developers to modify application logic or over-provision resources, resulting in compromised runtime efficiency for typical workloads such as microservices. We propose ServScale, a concurrency-aware serverless execution and scaling system that efficiently handling tradeoffs between intra-instance and inter-instance concurrency. ServScale introduces a hybrid scaling mechanism combining horizontal and vertical scaling, dynamically adapting to real-time workload characteristics. We design specialized prewarm and node scheduling strategies to minimize scaling latency while maintaining application QoS. Implemented on Kubernetes, ServScale reduces core-hour consumption by 20.3% compared to multi-threading enabled serverless systems using only horizontal scaling, and by 50.1% compared to traditional IaaS-based deployments.