A Linear Combination-Based Method to Construct Proxy Benchmarks for Big Data Workloads

During the early stages of CPU design, benchmarks can only run on simulators to evaluate CPU performance. However, most big data component benchmarks are unable to finish running on simulators at an acceptable time cost, as simulators are slower 100X–1000X times than physical platform. Moreover, big data benchmarks usually need the support of complex software stacks, which is hard to be ported on the simulators. Proxy benchmarks have the same micro-architectural characteristics as real benchmarks and do not require long running time or complex software stacks. Therefore, proxy benchmarks can replace real benchmarks to run on simulators. The biggest challenge of proxy benchmark generation is how to guarantee that the proxy benchmarks have exactly the same micro-architectural metrics as real benchmarks when the number of micro-architectural metrics is very large. To deal with this challenge, we propose a linear combination-based proxy benchmark generation methodology that transforms this problem into solving a system of linear equations. We also design the corresponding algorithms to ensure the system of linear equations is astringency. We generate fifteen proxy benchmarks and evaluate their running time and accuracy in comparison to the corresponding real benchmarks for MySQL and RockDB. On the typical Intel Xeon platform, the average running time is 1.62 s, and the average accuracy of every micro-architectural metric is over 92%, while the longest running time of real benchmarks is nearly 4 h. We also conduct two case studies that demonstrate that our proxy benchmarks are consistent with real benchmarks both before and after prefetch or Hyper-Threading is turned on.