Scholarlink

<p>Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to adapt to fluctuating workloads dynamically. This paper introduces BigOPERA, an opportunistic and elastic resource allocation framework designed to enhance BD processing environments by integrating dedicated and non-dedicated computing assets. Leveraging containerization and a two-tiered scheduling mechanism, BigOPERA dynamically manages available resources to improve workload execution efficiency. Experimental results demonstrate that BigOPERA achieves up to 35% performance improvement over native Apache Spark configurations, significantly enhancing computational throughput while optimizing resource consumption. Our findings highlight the potential of BigOPERA in scalable, cost-effective, and sustainable BD processing.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年6月16日

查看全文
  • Pablo V. Caderno,
  • Feras Awaysheh,
  • José C. Cabaleiro,
  • Tomás F. Pena

摘要

Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to adapt to fluctuating workloads dynamically. This paper introduces BigOPERA, an opportunistic and elastic resource allocation framework designed to enhance BD processing environments by integrating dedicated and non-dedicated computing assets. Leveraging containerization and a two-tiered scheduling mechanism, BigOPERA dynamically manages available resources to improve workload execution efficiency. Experimental results demonstrate that BigOPERA achieves up to 35% performance improvement over native Apache Spark configurations, significantly enhancing computational throughput while optimizing resource consumption. Our findings highlight the potential of BigOPERA in scalable, cost-effective, and sustainable BD processing.