Depression is a prevalent mental disorder worldwide, and its assessment remains challenging due to several subjective factors, thereby highlighting the necessity for data-driven approaches based on deep learning to enable more objective and effective detection. In this paper, Multi-modal Graph Diffusion for Depression (MGD4D), which integrates multi-modal learning with advanced graph-based diffusion modeling, is proposed for depression detection. In its novel framework, cerebral neuroimaging data and auxiliary information are represented as structural, functional, and informational subgraphs. Then the depression probability is inferred via an end-to-end pipeline comprising modality-specific encoders, a latent graph diffusion model, and a graph classification decoder. The key innovation of this diffusion model is its denoising network, which synergistically combines a Transformer-based multi-head attention mechanism with a U-Net module. In experimental validations, comparison experiments on publicly available datasets demonstrate that MGD4D significantly outperforms recent strong baselines in the detection of depression, achieving state-of-the-art (SOTA), while ablation studies further quantify the impact of each core component. Overall, this research provides a promising, objective, and intelligent approach for distinguishing between depressive patients and healthy controls (HCs), with substantial potential for adaptation to other mental disorders. The complete code is available at: https://github.com/RosalindFok/MGD4D.git .