The diversification of data modalities across various industries presents new challenges for time series prediction. Finance serves as a key example for this trend: quantitative data such as stock prices and financial indicators coexist with alternative data like sentiment embedded in news or social media. This raises a critical question: how can we fully harness the potential of this cross-modal data? In this paper, we propose a novel framework for cross-modal analysis, using finance as a representative case study. We explore two distinct approaches to integrating these data types. First, we quantify the alternative data modality by using a neural network to align text embeddings directly with market movements. Second, we introduce a dual-path transformer architecture designed to capture the cross-modal attention between quantitative market data and text-based financial news. Finally, we demonstrate the effectiveness of this cross-modal approach through comprehensive back-testing, where results show that the dual-path transformer, leveraging effectively both modalities, outperform models using purely quantitative data within our experimental framework. While the financial domain serves as our use case, the methodologies we develop are applicable to any scenario where multiple data modalities converge. By establishing this paradigm, we aim to provide key insights to this process.