MLDF-Net: Metadata Based Multi-level Dynamic Fusion Network

Computer-aided diagnosis has been widely used in the medical field, and one of the current research hotspots for aiding diagnosis is how to effectively fuse heterogeneous data such as image data and metadata. Most recent multi-modal skin cancer diagnosis models are only fused at the feature level or decision level and have not yet paid attention to the differential influence of metadata on image features under dynamic guidance, which has limited the ability of metadata to improve the predictive performance of the model. Therefore, this paper proposed a multi-level dynamic fusion network (MLDF-Net) based on metadata guidance, which attempted to dynamically fuse relevant metadata features in the image feature extraction stage to achieve the purpose of metadata-guided image features. Firstly, we designed a feature selection block (FS Block) to suppress the influence of noise in metadata and enhance the metadata feature representation associated with images. Secondly, the filtered metadata and images are fused in the feature extractor at multiple levels, and the metadata dynamically guides the network to extract more representative image features. Lastly, the experimental results showed that MLDF-Net achieved 81.3% accuracy compared with other classification studies using the same dataset, which verified the feasibility and advancement of the multi-level dynamic fusion strategy based on metadata guidance.