Performance optimization of heterogeneous computing for large-scale dynamic graph data

The performance of machines has not been fully exploited when processing large-scale dynamic graph in heterogeneous GPU clusters. To improve the performance of graph computing, a distributed heterogeneous engine (DHE) has been designed. A new heterogeneous graph partitioning algorithm is implemented, which can achieve load balancing among internal nodes in GPU clusters. DHE introduces synergy model to quantify the co-computing performance of heterogeneous processors and designs computing pipelines to optimize the performance of accessing memory. The graph algorithms PageRank, CC, SSSP and K-core are selected for the experiment. Under the same conditions, DHE’s partitioning algorithm improves scalability and load balancing ability compared to other graph partitioning algorithms. The heterogeneous computing pipeline exhibits better memory access compared to the basic engine. The experiments show that the synergy of this system converges to 1 stably. Compared with the other three graph partitioning algorithms, the system reduced processing time by approximately 20–30% and the performance is improved by a factor of 1.2 times.