Cross-dimensional knowledge-guided synthesizer trained with unpaired multimodality MRIs

Magnetic resonance images (MRIs) of different modalities have different reference values for pathological diagnosis. The difficulty of obtaining multimodal MRIs makes it effective to synthesize medical images of missing modalities from existing one. To train a one-for-all synthesizer with limited number of unpaired MRIs, a novel cross-dimensional knowledge-guided generative adversarial network (CKG–GAN) is proposed. In CKG–GAN, a cross-dimensional knowledge transfer network is utilized to measure the perceptual similarity of 2D images (slices of MRIs) from source and synthesized modalities, the knowledge of which is transferred from a pre-trained 3D network without accessing its private training data set. We evaluate the proposed model for three tasks on BraTs2018 and BraTs2021 data sets, using one modality of T1, T2 or Flair to synthesize the other two modalities without changing the content. The results show that compared with the current state-of-the-art methods, our method improves the performance by 2–7%.