In stream computing systems, fault tolerance and recovery efficiency during task execution are core elements for ensuring system performance. However, existing fault tolerance strategies often overemphasise global fault tolerance performance, leading to significant increases in resource overhead and failing to achieve an effective balance between resource costs and reliability. To address these issues, we propose a dynamic adaptive fault-tolerant strategy named Da-Stream. This paper addresses the following aspects: (1) The high resource costs associated with running both primary and backup copies simultaneously are analysed, and the impact of factors such as operator type, changes in data stream size, and the strength of upstream/downstream dependencies on the fault tolerance requirement level of operators is verified. (2) The stream computing resource model and operator fault tolerance requirement level model are established to evaluate node CPU and memory resource utilisation, satisfy the resource constraints of the fault tolerance strategy, and adaptively assess the fault tolerance requirement levels of different operators. (3) Da-Stream classifies strategies based on factors such as operator type, resource requirements, and dependencies, and dynamically adjusts the fault tolerance strategy at runtime by combining resource usage, fault prediction, and historical scores. (4) Experimental results show that compared with state-of-the-art methods, Da-Stream reduces fault recovery time by 24.3%, increases CPU utilisation by 18.7%, increases memory utilisation by 12.2%, and reduces processing latency by 36.8%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dynamic Adaptive Fault-Tolerance in Stream Computing Systems Under Resource Constraints

  • Zhaojun Wang,
  • Dawei Sun,
  • Xuan Zang,
  • Atul Sajjanhar,
  • Rajkumar Buyya

摘要

In stream computing systems, fault tolerance and recovery efficiency during task execution are core elements for ensuring system performance. However, existing fault tolerance strategies often overemphasise global fault tolerance performance, leading to significant increases in resource overhead and failing to achieve an effective balance between resource costs and reliability. To address these issues, we propose a dynamic adaptive fault-tolerant strategy named Da-Stream. This paper addresses the following aspects: (1) The high resource costs associated with running both primary and backup copies simultaneously are analysed, and the impact of factors such as operator type, changes in data stream size, and the strength of upstream/downstream dependencies on the fault tolerance requirement level of operators is verified. (2) The stream computing resource model and operator fault tolerance requirement level model are established to evaluate node CPU and memory resource utilisation, satisfy the resource constraints of the fault tolerance strategy, and adaptively assess the fault tolerance requirement levels of different operators. (3) Da-Stream classifies strategies based on factors such as operator type, resource requirements, and dependencies, and dynamically adjusts the fault tolerance strategy at runtime by combining resource usage, fault prediction, and historical scores. (4) Experimental results show that compared with state-of-the-art methods, Da-Stream reduces fault recovery time by 24.3%, increases CPU utilisation by 18.7%, increases memory utilisation by 12.2%, and reduces processing latency by 36.8%.