GQAT-Net: A Calibrated Attention Model for Failure Prediction in Large-Scale Distributed Systems
摘要
In large-scale cloud and IoT-enabled edge computing environments, ensuring high service availability and reliability is critical to minimizing task and job failures. Predicting these failures at the application level presents significant challenges due to the complexity, variability, and scale of workloads, compounded by resource constraints that are common in edge devices. In this paper, we introduce a novel framework, GQAT-Net: A Calibrated Grouped Query Attention Neural Network for Uncertainty-Aware Failure Prediction, which demonstrates strong potential for application in dynamic cloud and IoT-edge ecosystems. The proposed model integrates advanced attention mechanisms, multiscale gated convolutions, and transformer-based layers to capture temporal and spatial dependencies effectively, addressing the challenges of heterogeneous and dynamic workloads. In addition, calibration and uncertainty quantification techniques are employed to enhance the reliability of predictions, providing insights that could inform proactive resource and failure management strategies. The model is evaluated on two benchmark real world datasets: the Alibaba 2020 GPU Cluster Traces and the Google 2011 Cluster Traces. GQAT-Net achieves precision rates of 96.762% for task failures and 95.711% for job failures in the Alibaba dataset, and 98.383% for task failures and 97.873% for job failures in the Google dataset. These results highlight significant advancements in failure prediction accuracy, demonstrating the model’s potential to enhance reliability and optimize resource efficiency in large-scale distributed systems. The code for our proposed model is made public and can be accessed at the following url: https://github.com/tri-shi/GQAT-NET.git.