Language Model-Guided Visual Reasoning for Scene-Aware Pedestrian Intention Prediction
摘要
Predicting pedestrian crossing intention is vital for enabling safe and anticipatory behaviour in autonomous vehicles (AVs), particularly in complex and visually degraded urban settings. Existing approaches often rely on high-quality visual inputs, dense semantic maps, and computationally intensive architectures, limiting their generalizability and real-time applicability. To address these limitations, we propose an end-to-end framework that unifies visual appearance, motion dynamics, and symbolic scene attributes for robust intention forecasting. Our method integrates a MobileViT-S backbone for visual encoding, a GRU-based trajectory encoder for motion, and a template-driven semantic context encoder whose outputs are embedded via MiniLM-L6-v2. These heterogeneous cues are fused through an Attention-Gated Cross-Modal Fusion Block that adaptively aligns modalities via cross-attention and gating mechanisms. Evaluated on the PIE and JAAD datasets, our model achieves competitive results (AUC: 0.92 on PIE, 0.89 on JAAD) using only ~0.99M trainable parameters.