Multi-modal Recurrent Graph Neural Networks for Spatiotemporal Forecasting

The spatial and temporal dynamics of many real-world systems present a significant challenge to multi-variate forecasting where features of both forms, as well as their inter-dependencies, must be modeled correctly. State-of-the-art approaches utilize a limited set of exogenous features (outside the forecast variable) to model temporal dynamics and Graph Neural Networks, with pre-defined or learned networks, to model spatial dynamics. While much work has been done to model dependencies, existing approaches do not adequately capture the explicit and implicit modalities present in real-world systems. To address these limitations we propose MMR-GNN, a spatiotemporal model capable of (a) augmenting pre-defined (or absent) networks into optimal dependency structures (b) fusing multiple explicit modalities and (c) learning multiple implicit modalities. We show improvement over existing methods using several hydrology and traffic datasets. Our code is publicly available at https://github.com/HipGraph/MMR-GNN .