MGCAF: a multi-granularity cross-modal alignment framework for grounded multimodal named entity recognition
摘要
Grounded multimodal named entity recognition (GMNER) extends multimodal named entity recognition (MNER) by linking textual entities with corresponding visual regions. Existing approaches face two main challenges: (1) they often focus on either global or local visual features, where the former misses fine-grained correspondences and the latter overlooks broader contextual cues; (2) current methods lack unified and semantically aligned representations, limiting cross-modal consistency. To address these issues, we propose a multi-granularity cross-modal alignment framework (MGCAF), which leverages global and local visual features along with textual cues from images, enabling multi-granularity cross-modal alignment and interactions between text and images. Specifically, we introduce a multi-granularity visual feature fusion (MVF) module to balance and integrate global and local visual information and a multi-granularity cross-modal alignment (MCA) module to align multi-granularity visual and textual representations. Notably, GMNER involves large-scale multimodal processing and dense cross-modal interactions, and the proposed MGCAF is inherently suited for parallel and distributed computation, enabling efficient training and scalable inference. Experimental results demonstrate that MGCAF improves both entity recognition and visual grounding, validating its effectiveness for the GMNER task.