Deep Reinforcement Learning (DRL) has achieved significant progress in various domains, notably in autonomous systems and strategic gameplay, exemplified by developments such as AlphaGo and the MuZero algorithm series. Despite these advancements, the considerable computational demands of DRL models present challenges for their practical implementation in real-world applications. This research paper addresses the issue of hyperparameter prediction within the context of DRL training. We introduce HGAPerf, a method designed to forecast the run time during the training of DRL models. HGAPerf integrates heterogeneous graph hyperparameter mapping with predictive modeling, employing Graph Neural Networks (GNNs) enhanced by multi-head attention mechanisms. This approach aims to improve the accuracy of training time predictions for the MuZero series algorithm. To evaluate the performance of HGAPerf, we conducted a series of experiments, including baseline comparisons and an ablation study. The results indicate that HGAPerf can improve runtime prediction accuracy and enhance training stability. In comparative performance evaluations, HGAPerf showed improvements over existing models such as DNNPerf and Population Based Training (PBT). The model achieved a Mean Relative Error (MRE) of 13.4% and a Root Mean Square Error (RMSE) of 77.4 ms.