Rethinking a Unified Generative Adversarial Model for MRI Modality Completion

Multi-modal MRIs are essential in medical diagnosis; however, the problem of missing modalities often occurs in clinical practice. Although recent works have attempted to extract modality-invariant representations from available modalities to perform image completion and enhance segmentation, they neglect the most essential attributes across different modalities. In this paper, we propose a unified generative adversarial network (GAN) with pairwise modality-shared feature disentanglement. We develop a multi-pooling feature fusion module to combine features from all available modalities, and then provide a distance loss together with a margin loss to regularize the symmetry of features. Our model outperforms the existing state-of-the-art methods for the missing modality completion task in terms of the generation quality in most cases. We show that the generated images can improve brain tumor segmentation when the important modalities are missing, especially in the regions which need details from various modalities for accurate diagnosis.