Multimodal Imaging Classification Based on Mixture Model Networks

In medical imaging, different modality reflect different clinical information due to their different imaging principles. Compared with single-modality imaging analysis, multimodal fusion technology can make use of complementary information among modalities. To capture the potential relationships of different modalities, this paper proposes a multimodal imaging classification framework based on Mixture model Networks (MoNet). Firstly, from the perspective of graph structure, the feature is regarded as a node. The K-nearest neighbor algorithm is used to construct initial image representation from the features extracted from multimodal imaging, and the Graclus algorithm is used to achieve multiple feature dimensionality reduction. Then, MoNet is used to realize the adaptive updating of edge weights and mine the potential connections between features. Finally, the Fully Connected Network is employed to complete the prediction classification. Comparative experiments are carried out on the BRATS2017 dataset, and the results show that our method provides better performance than the baseline.