Fine-Grained Medical Image Synthesis with Dual-Attention Adversarial Learning

Medical imaging plays a critical role in various clinical applications. However, due to considerations such as cost and risk, the acquisition of certain image modalities can be limited. To address this issue, many cross-modality medical image synthesis methods have been proposed. Nevertheless, current methods struggle to accurately model hard-to-synthesize regions (e.g., tumor or lesion regions). To overcome this challenge, we propose a simple yet effective strategy: a dual-discriminator (dual-D) adversarial learning system. In this system, 1) a global discriminator (global-D) provides an overall evaluation of the synthetic image, and 2) a local discriminator (local-D) performs a dense evaluation of the synthetic image’s local regions. Additionally, we introduce a difficult-region-aware attention mechanism that enhances the modeling of hard-to-synthesize regions (e.g., tumor or lesion regions) based on the local-D. Experimental results demonstrate the robustness and accuracy of our proposed method in synthesizing target images from corresponding source images. Specifically, we evaluated our method on two datasets: i.e., 1) generating T2 MRI from T1 MRI for brain tumor images, and 2) generating CT from MRI. Our proposed method outperforms state-of-the-art techniques in both datasets and tasks. Furthermore, our proposed difficult-region-aware attention mechanism proves effective in generating more realistic images, particularly in the hard-to-synthesize regions.