Scholarlink

<p>Transfer learning from pre-trained models is a powerful technique, but selecting the best performing models for a new task with limited data can be challenging. This paper presents a novel approach for predicting the performance ranking of pre-trained models in low-data scenarios. It has the potential to significantly streamline model selection, enabling practitioners to maximize performance while minimizing resource consumption. Note that, this work presents a transferability prediction metric optimized for pre-trained models operating on domain-specific structured data, exemplified by image datasets, and should not be generalized to unstructured or arbitrary tabular formats. The proposed method employs a two-step process. First, it estimates the expected performance and associated variance of each candidate model by leveraging multi-resolution K-nearest neighbors (KNN) combined with resampling techniques on the features extracted from the target dataset using the candidate model. Second, it utilizes Hotelling’s <i>T</i>-squared test to determine whether the performance differences between consecutively ranked models are statistically significant. The effectiveness of the proposed approach is evaluated using stratified downsampling to simulate low-data scenarios. In particular, our proposed methods, BiMod and SimBoot, significantly advance Transferability prediction. They achieve high computational efficiency, running up to 3x faster than LogMe and orders of magnitude faster than Transrate. Crucially, they also demonstrate superior predictive performance, consistently yielding the highest weighted Kendall’s tau scores across diverse low-data scenarios (e.g., up to 0.876 for CIFAR10 and 0.869 for CIFAR100), outperforming all benchmarks with remarkable stability.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Transferability prediction in sparse data regime using multi-resolution KNN predictor

  • 期刊论文

  • 发表日期：

    2025年9月13日

查看全文
  • Napa Sae-Bae,
  • Sirisup Laohakiat

摘要

Transfer learning from pre-trained models is a powerful technique, but selecting the best performing models for a new task with limited data can be challenging. This paper presents a novel approach for predicting the performance ranking of pre-trained models in low-data scenarios. It has the potential to significantly streamline model selection, enabling practitioners to maximize performance while minimizing resource consumption. Note that, this work presents a transferability prediction metric optimized for pre-trained models operating on domain-specific structured data, exemplified by image datasets, and should not be generalized to unstructured or arbitrary tabular formats. The proposed method employs a two-step process. First, it estimates the expected performance and associated variance of each candidate model by leveraging multi-resolution K-nearest neighbors (KNN) combined with resampling techniques on the features extracted from the target dataset using the candidate model. Second, it utilizes Hotelling’s T-squared test to determine whether the performance differences between consecutively ranked models are statistically significant. The effectiveness of the proposed approach is evaluated using stratified downsampling to simulate low-data scenarios. In particular, our proposed methods, BiMod and SimBoot, significantly advance Transferability prediction. They achieve high computational efficiency, running up to 3x faster than LogMe and orders of magnitude faster than Transrate. Crucially, they also demonstrate superior predictive performance, consistently yielding the highest weighted Kendall’s tau scores across diverse low-data scenarios (e.g., up to 0.876 for CIFAR10 and 0.869 for CIFAR100), outperforming all benchmarks with remarkable stability.