The proliferation of AI-synthesized images and text online has raised significant security concerns, underscoring the need for robust detection and grounding of multi-modal manipulation. Existing methods primarily rely on vision-language pre-training paradigms, and frequently lack specialized interaction designs for multi-modal manipulation, failing to simultaneously address the different requirements of images and text for representational coordination. In this paper, we propose a novel modality interaction framework incorporating patch-gated cross-attention (PGCA) and multi-level feature injection (MLFI). Specifically, PGCA employs a patch-level gating mechanism during modality interactions to filter cross-modal features while preserving crucial image-specific features. MLFI injects multi-level semantic knowledge into the interaction layer to capture subtle discrepancies between images and text. Extensive experiments on the DGM4 and Fakeddit datasets demonstrate that our approach outperforms state-of-the-art methods.