Many modern cloud applications require intra-instance concurrency to efficiently handle multiple requests or tasks in parallel. While platforms like Kubernetes are widely adopted for scalable deployment on Infrastructure-as-a-Service (IaaS), this approach often leads to resource inefficiencies due to data center tax and idle containers. Serverless computing provides elastic scaling and pay-as-you-go benefits, but migrating such applications remains challenging. Most function containers of existing serverless platforms process requests serially and rely solely on horizontal scaling, forcing developers to modify application logic or over-provision resources, resulting in compromised runtime efficiency for typical workloads such as microservices. We propose ServScale, a concurrency-aware serverless execution and scaling system that efficiently handling tradeoffs between intra-instance and inter-instance concurrency. ServScale introduces a hybrid scaling mechanism combining horizontal and vertical scaling, dynamically adapting to real-time workload characteristics. We design specialized prewarm and node scheduling strategies to minimize scaling latency while maintaining application QoS. Implemented on Kubernetes, ServScale reduces core-hour consumption by 20.3% compared to multi-threading enabled serverless systems using only horizontal scaling, and by 50.1% compared to traditional IaaS-based deployments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ServScale: Concurrency-Aware Serverless Execution and Scaling Paradigm

  • Zichen Xu,
  • Zijun Li,
  • Quan Chen,
  • Minyi Guo

摘要

Many modern cloud applications require intra-instance concurrency to efficiently handle multiple requests or tasks in parallel. While platforms like Kubernetes are widely adopted for scalable deployment on Infrastructure-as-a-Service (IaaS), this approach often leads to resource inefficiencies due to data center tax and idle containers. Serverless computing provides elastic scaling and pay-as-you-go benefits, but migrating such applications remains challenging. Most function containers of existing serverless platforms process requests serially and rely solely on horizontal scaling, forcing developers to modify application logic or over-provision resources, resulting in compromised runtime efficiency for typical workloads such as microservices. We propose ServScale, a concurrency-aware serverless execution and scaling system that efficiently handling tradeoffs between intra-instance and inter-instance concurrency. ServScale introduces a hybrid scaling mechanism combining horizontal and vertical scaling, dynamically adapting to real-time workload characteristics. We design specialized prewarm and node scheduling strategies to minimize scaling latency while maintaining application QoS. Implemented on Kubernetes, ServScale reduces core-hour consumption by 20.3% compared to multi-threading enabled serverless systems using only horizontal scaling, and by 50.1% compared to traditional IaaS-based deployments.