The booming civil aviation industry has brought increasingly growing flight delays, which cause significant losses to airlines, airports, and passengers. Various factors affecting the delays, including the airport network structure, the number of scheduled, delayed, and cancelled flights in each airport, the weather and special situation information at each airport, etc. It is challenging to comprehensively utilize the information above and the existing methods predict flight delays based on only part of the airport network structure information, which is hard to effectively extract the nonlinear delay features. In this paper, we propose a Multi-Relationships Diffusion based Graph Convolutional Network (MRD-GCN) which captures the propagation pattern of flight delays by incorporating various delay related factors for flight delay forecasting. MRD-GCN extracts and accumulates multiple airport relationships, such that the edges in the airport network graph contains more flight delay propagation information in the spatial domain. In addition, MRD-GCN enriches the airport node information in the time domain by taking the dynamic weather and special situation of the airports as supplementary information to the original node in the airport network graph. We conduct single-step and multi-step experiments on real flight delay data. The experimental results show that MRD-GCN can effectively improve the flight delay prediction accuracy.