Reimagining Attentional Copulas: A Transformer-Based Approach with Proportional Dependency Learning

Multivariate probabilistic time series forecasting is crucial for decision-making tasks that rely on future values, uncertainty quantification, and inter-variable dependencies. Recently, attentional copulas have gained popularity for successfully combining the expressiveness of deep learning with the modular structure of copula models, which separate the modeling of marginals from that of the joint distribution. Despite their promise, existing methods such as TACTiS face challenges related to convergence stability and the accurate calibration of dependencies. We propose T-PAKC, a Transformer-based Proportional Attentional Kernelized Copula model to address these challenges through: (1) replacing discrete copula binning with a Gaussian kernelized estimator for smooth gradients and expressive density modeling, and (2) adding a Proportional Absolute Error (PAE) regularizer to improve joint calibration without harming marginals. Experiments on five real-world datasets show T-PAKC consistently outperforming strong baselines in CRPS and Energy Scores, with more stable convergence. This work offers a robust, domain-agnostic framework that combines statistical rigor with deep learning flexibility, advancing scalable, uncertainty-aware probablistic multivariate forecasting.