Optimization of maximum parallel K-means algorithm based on GPU cluster
摘要
When considering the data computing process, computing model, CPU acceleration and other aspects, it can be found that when large-scale data needs parallel computing, the overhead of K-Means filtering algorithm in the data filtering phase and the cluster center update phase running on the CPU will soon offset the efficiency improvement due to the reduction of distance computing. In view of the disadvantage that the filtering algorithm is not widely scenario oriented, this paper maximizes the parallel optimization of K-Means algorithm. Convert each step in the algorithm iteration part into a kernel function, discuss the impact of the task amount allocated to each thread on the overall efficiency of the algorithm, and optimize the design and implementation of data point allocation and cluster center update. After selecting the sub stage algorithm with the best performance, the fusion algorithm is formed and implemented on the GPU cluster. The fusion algorithm is tested under multiple parameter ranges by changing the data set size, clustering number, data dimension and other important parameters. The experimental results show that the fusion algorithm combined after optimizing each step of the K-Means algorithm has obvious acceleration effect in all parameter ranges compared with the benchmark algorithm and the filtering algorithm. Compared with the K-Means filtering algorithm, the computing speed has been improved by more than ten times, and the best overall performance can be achieved in the CPU-GPU heterogeneous cluster system.