Enhancing Temporal Transformers for Financial Time Series via Local Surrogate Interpretability

The advent of Transformer architectures has ushered in a new era across various domains, including finance. These Transformer models, renowned for their scalability and efficacy, are considered opaque due to their black-box nature. The inherent complexity of these models, despite the partial interpretability afforded by the attention mechanism, poses a significant barrier to non-technical stakeholders in finance. This is challenging, not just for investors struggling to interpret the models’ outputs, but also for technical experts who aspire to refine these models. In this research, we introduce a novel approach that leverages the interpretability of local surrogate models to enhance complex temporal Transformer models in financial forecasting. Utilizing insights from the surrogate models, we iteratively improve the model’s predictive performance in both volatile and stable market conditions through informed feature selection. This contribution combines the simplicity of local surrogate models with the robust capabilities of temporal Transformers. Our experimental results demonstrate notable improvements in forecasting accuracy during volatile periods, with equally promising improvements in the non-volatile phases. These findings suggest that integrating explainability into the training process of financial models not only aids in demystifying their operation but also significantly bolsters their performance. Our solution leverages an emerging paradigm in machine learning where explainability becomes an integral component of model development, paving the way for more robust and intuitive financial forecasting models.