Scholarlink

<p>Deep learning has led to significant state-of-the-art results in a panoply of fields with its pattern recognition ability. Even though the research community has strongly benefited from this, these models showcase characteristics that hinder real-world application. One of the major hurdles is the difficulty in pinpointing what a neural network does not know along with their generalisation capabilities. In this context, the term underspecification has been coined, which describes the generation of different predictors with similar in-domain accuracy but diverging results in OOD. In this paper, we characterise the underspecification distribution and study its connection with epistemic uncertainty. We propose the average-metric epistemic uncertainty that transforms the epistemic uncertainty to the underspecification space. We perform a set of experiments using both LeNet and ResNet18 to solve classification problems on CIFAR-10 and Tiny-ImageNet, respectively. We verify that the average-metric epistemic uncertainty is able to accurately predict, on average, 95% of the predictors that can be obtained from a single architecture. In order to improve the interpretability of neural networks, we suggest utilising the range estimated by the average-metric epistemic uncertainty alongside the accuracy to characterise future state-of-the-art models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Underspecification and uncertainty in deep learning models: Is there a connection?

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月11日

查看全文
  • Filipa M. M. Ramos Ferreira,
  • Rosaldo J. F. Rossetti

摘要

Deep learning has led to significant state-of-the-art results in a panoply of fields with its pattern recognition ability. Even though the research community has strongly benefited from this, these models showcase characteristics that hinder real-world application. One of the major hurdles is the difficulty in pinpointing what a neural network does not know along with their generalisation capabilities. In this context, the term underspecification has been coined, which describes the generation of different predictors with similar in-domain accuracy but diverging results in OOD. In this paper, we characterise the underspecification distribution and study its connection with epistemic uncertainty. We propose the average-metric epistemic uncertainty that transforms the epistemic uncertainty to the underspecification space. We perform a set of experiments using both LeNet and ResNet18 to solve classification problems on CIFAR-10 and Tiny-ImageNet, respectively. We verify that the average-metric epistemic uncertainty is able to accurately predict, on average, 95% of the predictors that can be obtained from a single architecture. In order to improve the interpretability of neural networks, we suggest utilising the range estimated by the average-metric epistemic uncertainty alongside the accuracy to characterise future state-of-the-art models.