Recent advances in deep learning, particularly transformer-based models, have shown remarkable success in capturing both spatial and temporal dependencies in video and image data. Interestingly, financial time series data share analogous structures: asset returns at different time points form sequences, and the covariance matrix captures inter-asset relationships, similar to spatial correlations in images. Despite this analogy, existing financial forecasting models rarely leverage these spatio-temporal patterns to their full potential. This research addresses this gap by leveraging the Time-Space Transformer (TimeSformer) model, originally developed for video understanding, to analyze and predict financial time series data. Specifically, we transform the return features of portfolio stocks into image representations with a temporal structure. We experiment with four different TimeSformer architectures, employing both parametric and non-parametric loss functions to predict the subsequent image, which is then back-transformed into the corresponding returns and covariance matrices. Experimental results demonstrate that this approach greatly improves prediction accuracy, achieving lower RMSE for returns and reduced log-euclidean distance for covariance matrices compared to traditional methods.