The demand for environmentally friendly cloud computing is on the rise, leading cloud service providers to focus on reducing carbon emissions by using renewable energy sources and energy-efficient computing models. This study assesses the performance and energy consumption of serverless and serverful architectures, specifically looking at join operations using Apache Spark for big data processing in a private cloud combined with Kubernetes. By using the TPC-DS benchmark, we examine the impact of cold-start and warm-start phases in the serverless environment, as well as the auto-scaling capabilities of Spark in serverless environments within the private cloud. The results show that the efficient and flexible resource management in serverless environments in private clouds leads to more optimal processing times and energy consumption compared to serverful architectures, especially in warm-start scenarios. These findings offer valuable insights for organizations seeking to streamline their big data infrastructure while also making a positive environmental impact within the IT industry.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Energy and Performance Evaluation of Serverless and Serverful Models on Spark for Database Join Operations

  • Phan-An-Truong Tran,
  • Laurent D’orazio,
  • Thuong-Cang Phan,
  • Le Gruenwald

摘要

The demand for environmentally friendly cloud computing is on the rise, leading cloud service providers to focus on reducing carbon emissions by using renewable energy sources and energy-efficient computing models. This study assesses the performance and energy consumption of serverless and serverful architectures, specifically looking at join operations using Apache Spark for big data processing in a private cloud combined with Kubernetes. By using the TPC-DS benchmark, we examine the impact of cold-start and warm-start phases in the serverless environment, as well as the auto-scaling capabilities of Spark in serverless environments within the private cloud. The results show that the efficient and flexible resource management in serverless environments in private clouds leads to more optimal processing times and energy consumption compared to serverful architectures, especially in warm-start scenarios. These findings offer valuable insights for organizations seeking to streamline their big data infrastructure while also making a positive environmental impact within the IT industry.