Convolutional Cross-Modal Autoencoder-Based Few-Shot Learning for Data Augmentation with Application to Alzheimer Dementia Diagnosis

This paper presents a novel deep few-shot learning method for magnetic resonance images-based Alzheimer’s dementia (AD) diagnosis. The proposed method consists of two main phases namely data augmentation and data classification. With regard to data augmentation and, to deal with data scarcity issues, we designed a convolutional cross-modal autoencoder (CCMAE) model for data generation. This model, which consists of two encoders and one decoder, receives two image modalities namely longitudinal and cross-section MRI, and generates a new cross-section image. We opt for a convolutional version of the autoencoder to capture the spatial information more effectively and reduce the number of trainable parameters. Moreover, to make the model able to perform deep analysis of input image, we establish a skip connection strategy between the first encoder and the decoder similar to the UNet mechanism. With regard to classification, we design a convolutional neural network-based model in which both textual and visual features are fused to strengthen the network performance and produce more reliable decisions. A comprehensive experiment on a publicly available dataset has been conducted to demonstrate the effectiveness of the proposed method compared to some related works. The code is publicly available at: https://github.com/Bazine-Othmane/scientific-paper-code.