Benchmarking time series models is challenging due to non-stationarity, irregular sampling, horizon-dependent behaviour, and structural variability across domains. This chapter develops a principled framework for evaluating temporal models, integrating classical error measures with metrics that quantify temporal alignment, distributional fidelity, and dynamical consistency. We review techniques ranging from Dynamic Time Warping, Fréchet and Wasserstein distances, and energy statistics to continuous-time criteria derived from stochastic processes and differential-equation-based dynamics. Beyond individual metrics, we examine evaluation protocols–including rolling-window, expanding-window, and multi-horizon schemes–that characterize generalization under realistic deployment scenarios. We also survey benchmark datasets designed to probe diverse temporal regimes, from high-frequency signals to sparse, event-based processes. By grounding these methods in statistical theory and dynamical systems analysis, the chapter articulates a transparent and reproducible benchmarking philosophy that strengthens comparative evaluation and advances the scientific maturity of continuous-time forecasting.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Model Evaluation and Benchmarking

  • Mansura Habiba,
  • Barak A. Pearlmutter,
  • Mehrdad Maleki

摘要

Benchmarking time series models is challenging due to non-stationarity, irregular sampling, horizon-dependent behaviour, and structural variability across domains. This chapter develops a principled framework for evaluating temporal models, integrating classical error measures with metrics that quantify temporal alignment, distributional fidelity, and dynamical consistency. We review techniques ranging from Dynamic Time Warping, Fréchet and Wasserstein distances, and energy statistics to continuous-time criteria derived from stochastic processes and differential-equation-based dynamics. Beyond individual metrics, we examine evaluation protocols–including rolling-window, expanding-window, and multi-horizon schemes–that characterize generalization under realistic deployment scenarios. We also survey benchmark datasets designed to probe diverse temporal regimes, from high-frequency signals to sparse, event-based processes. By grounding these methods in statistical theory and dynamical systems analysis, the chapter articulates a transparent and reproducible benchmarking philosophy that strengthens comparative evaluation and advances the scientific maturity of continuous-time forecasting.