A Novel Approach to Trajectory Situation Awareness Using Multi-modal Deep Learning Models

This paper presents a novel multi-modal deep learning framework, TSA-MM, for accurate trajectory situational awareness in transportation and military systems. The proposed framework integrates multiple sources of data, including trajectory data, USNI NEWS data, and track situation maps, and effectively addresses the challenges of data heterogeneity, data sparsity, and model interpretability. The proposed approach combines visual, textual, and numerical information to predict the future trajectory of aircraft and achieves state-of-the-art performance. Notably, the proposed framework balances the importance and relevance of different tasks, demonstrating its flexibility and adaptability to various scenarios. The experiments conducted in this study demonstrate the accuracy and real-time performance of the proposed framework, highlighting its potential for trajectory situational awareness. Furthermore, this paper emphasizes the importance of developing more effective and interpretable models that can handle the complexity and heterogeneity of trajectory data and integrate domain knowledge into the model. Overall, the proposed multi-modal deep learning framework provides a promising approach for trajectory situational awareness and contributes to the development of more advanced and practical TSA methods.