MultiRoiMix: A Data Augmentation Method for PET/CT Multimodal Medical Images

This study proposed MultiRoiMix, a data augmentation method for multi-modal medical images, aiming to explore the potential of multi-modal data augmentation techniques in multi-modal segmentation tasks.

MultiRoiMix augmented images by incorporating both radiopharmaceutical metabolism information from PET images and anatomical location information from CT images. Our approach combined labeled data from both modalities to generate new samples. With regard to the generation strategy, MultiRoiMix integrated regions of interest from CT and PET images and established cross-modal information sharing links to create training samples with corresponding images and annotations.

The results showed that our approach achieved 84.18% performance (Dice: 55.45) with less than 1/3 of the training data compared to using 100% of the available data, thus addressing concerns regarding inadequate training due to sparse multi-modal datasets.

We validated our method in multimodal non-small cell lung cancer segmentation tasks using PET/CT multimodal datasets, and the results demonstrated the efficacy of our approach in multimodal segmentation networks.