Adaptive Multimodal Deep Networks for Climate Change Impact Prediction on Agriculture
摘要
In this work, we present the Adaptive Multimodal Attention Network (AMAN), a new deep learning model for resilient, scalable and fair prediction of climate change impacts on agriculture. Climate change poses a dire threat to food security, but impacts vary across agro-ecological regions, which existing unimodal, non-adaptive models do not address. AMAN overcomes these challenges by integrating satellite, climate and geospatial soil data using a collection of modality-specific encoders (Spatiotemporal Vision Transformer, Temporal Fusion Transformer, and Graph Attention Network) and a domain-adaptive cross-modal attention network. An adaptive learning framework involves two steps: pre-training a foundation model on abundant data and fast-adapting through Model-Agnostic Meta-Learning (MAML). AMAN is evaluated on five target agricultural regions, and outperforms the baselines: crop yield MAE is 8.3% lower, drought stress RMSE 11.6% lower, and 9.4% better cross-region generalization in the low-data target region. The key insight is that MAML adaptation for domain transfer significantly improves over fine-tuning in low-data regions, reaching comparable performance with only 10 target labels, compared to 50 + for the best baseline. AMAN generalizes to 500 + diverse agricultural regions with low inference costs (2x). Attention and SHAP analyses show that AMAN pinpoints the climatic and edaphic factors driving vulnerability, enabling practical, climate-smart agricultural planning and food security strategies.