Current computer vision foundation models focus on general-purpose tasks but have limited specialized spatiotemporal reasoning capabilities. Adapting these models to geospatial applications and multimodal workflows requires addressing core challenges in spatial, temporal, and spectral representation learning. This paper proposes methodologies and a series of experiments aimed at advancing foundation models with spatiotemporal reasoning capabilities tailored for multimodal geospatial applications. These experiments address local and global position encoding, multi-resolution and multi-sensor fusion for remote sensing data, temporal reasoning, and multimodal fusion of additional spatiotemporal data types, including geospatial vectors and motion trajectories. The goal is to develop robust embedding strategies and model architectures that can generalize across regions, sensors, modalities, and timescales for downstream applications requiring spatiotemporal understanding.