Multi-fidelity surrogate models are commonly used in industries such as aerospace, marine engineering, and bioprocessing, where high-fidelity data is limited, as collecting such data is time-consuming or expensive. To supplement high-fidelity data, low-fidelity data, which is faster and cheaper to collect but less accurate, is often used. However, not all low-fidelity data necessarily improves model performance. In this research, we examine the sample spaces of high- and low-fidelity data with different distributions to identify those subsets of low-fidelity data that enhance model performance. Our hypothesis is that choosing low-fidelity data that is ‘more similar’ to high-fidelity data will result in better performance. The question is how to define similarity here. To explore this, we introduce two novel methods: a distance-based measure and a rank-based method, i.e. Kendall’s tau to quantify similarity and determine the best low-fidelity subsets for further investigation. A non-parametric permutation method for Hotelling’s \(T^2\) is then applied to assess the similarity between selected low-fidelity subsets and the high-fidelity data. We assess the effectiveness of these methods in identifying more informative low-fidelity subsets and their impact on the performance of multi-fidelity models, i.e. Autoregressive Gaussian Process and Deep Gaussian Process. We conduct a comprehensive evaluation of two selection strategies on both synthetic functions and a real-world bioprocessing case study. The experimental results further support our hypothesis. In summary, this research provides valuable insights into effectively identifying the subsets of low-fidelity data that could enhance the performance of multi-fidelity models.