<p>We examine the problem of handling uncertainty in the predictions for out-of-distribution data (OOD) when classifying time series with distortions, such as gaps, that can occur due to operating conditions in production environments. This problem can find several application domains, including predictive maintenance. We focus on Conformal Prediction (CP) as a framework for handling the uncertainty in predictions for OOD time series that occurs due to distortions. Our study focuses on the potential impact of OOD time series on the performance of CP, by assessing the size and coverage of the resulting prediction sets. The motivation for this study is that neural networks, which are widely used for time-series classification, may suffer from overconfidence. This fact negatively impacts CP when faced with OOD data, because incorrect predictions with high confidence can have catastrophic consequences in high-risk applications. To alleviate this problem, we propose two model-agnostic methods: the use of various forms of label smoothing as well as the use of hybrid classifiers. Our experimental findings in the context of prominent time-series classifiers demonstrate that the coverage of CP may be maintained around a desirable level without needlessly expanding the size of the prediction sets.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Conformal prediction for out-of-distribution time-series classification

  • Alexandros Nanopoulos,
  • Krisztian Buza

摘要

We examine the problem of handling uncertainty in the predictions for out-of-distribution data (OOD) when classifying time series with distortions, such as gaps, that can occur due to operating conditions in production environments. This problem can find several application domains, including predictive maintenance. We focus on Conformal Prediction (CP) as a framework for handling the uncertainty in predictions for OOD time series that occurs due to distortions. Our study focuses on the potential impact of OOD time series on the performance of CP, by assessing the size and coverage of the resulting prediction sets. The motivation for this study is that neural networks, which are widely used for time-series classification, may suffer from overconfidence. This fact negatively impacts CP when faced with OOD data, because incorrect predictions with high confidence can have catastrophic consequences in high-risk applications. To alleviate this problem, we propose two model-agnostic methods: the use of various forms of label smoothing as well as the use of hybrid classifiers. Our experimental findings in the context of prominent time-series classifiers demonstrate that the coverage of CP may be maintained around a desirable level without needlessly expanding the size of the prediction sets.