Scholarlink

<p>Evaluating and documenting the robustness of forecasting models to different input conditions is important for their responsible deployment in real-world applications. Time series forecasting models often exhibit degraded performance in the form of unusually large errors, high uncertainty, or hubris (high errors coupled with low uncertainty). Traditional stress testing approaches rely on manually designed adverse scenarios that fail to systematically identify unknown stress factors, in which data characteristics indicate potential issues. To overcome this limitation, this paper introduces <Emphasis FontCategory="NonProportional">MAST</Emphasis> (Meta-learning and data Augmentation for Stress Testing), a novel method for stress testing forecasting models. <Emphasis FontCategory="NonProportional">MAST</Emphasis> leverages model outputs (error scores and prediction intervals) to automatically identify and characterize input conditions that induce stress. Specifically, <Emphasis FontCategory="NonProportional">MAST</Emphasis> is a binary probabilistic classifier that predicts the likelihood of forecasting model stress based on time series features. An additional contribution is a novel time series data augmentation approach based on oversampling or synthetic time series generation, that improves the information about stress factors in the input space, resulting in increased stress classification performance. Experiments were conducted using 6 benchmark datasets containing a total of 97.829 time series. We demonstrate how <Emphasis FontCategory="NonProportional">MAST</Emphasis> is able to identify and explain input conditions that lead to manifestations of stress, namely large errors, high uncertainty, or hubris.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Mast: interpretable stress testing via meta-learning for forecasting model robustness evaluation

  • 期刊论文

  • 发表日期：

    2025年10月15日

查看全文
  • Ricardo Inácio,
  • Vitor Cerqueira,
  • Marília Barandas,
  • Carlos Soares

摘要

Evaluating and documenting the robustness of forecasting models to different input conditions is important for their responsible deployment in real-world applications. Time series forecasting models often exhibit degraded performance in the form of unusually large errors, high uncertainty, or hubris (high errors coupled with low uncertainty). Traditional stress testing approaches rely on manually designed adverse scenarios that fail to systematically identify unknown stress factors, in which data characteristics indicate potential issues. To overcome this limitation, this paper introduces MAST (Meta-learning and data Augmentation for Stress Testing), a novel method for stress testing forecasting models. MAST leverages model outputs (error scores and prediction intervals) to automatically identify and characterize input conditions that induce stress. Specifically, MAST is a binary probabilistic classifier that predicts the likelihood of forecasting model stress based on time series features. An additional contribution is a novel time series data augmentation approach based on oversampling or synthetic time series generation, that improves the information about stress factors in the input space, resulting in increased stress classification performance. Experiments were conducted using 6 benchmark datasets containing a total of 97.829 time series. We demonstrate how MAST is able to identify and explain input conditions that lead to manifestations of stress, namely large errors, high uncertainty, or hubris.