Research on Feature Fusion Methods for Multimodal Medical Data

With the rapid development of artificial intelligence, knowledge graph, image processing, etc. have been widely used, and smart medical care, as a major application scenario of artificial intelligence, has received a lot of attention. Traditional diagnostic methods have problems such as low accuracy and low efficiency, and the research and application of knowledge graph and image classification in the field of dermatology are also in the initial stage, but text-based knowledge graph technology and image-based image classification technology have developed very maturely. Considering that various current image classification algorithms extract features, feature calculation, and model matching from images, they do not consider obtaining information such as features or relationships that are not in images from text data to participate in image classification tasks. In this paper, the optimized hierarchical perception model H-HAKE based on hierarchical perception model KGE-HAKE calculates selector parameters by improving the hierarchical perception model to add category dimension to the TransE coordinate system, divide more image features and entities with the same attribute into the same level, increase the number of links between image and map entities, and produce better data coverage effect. Aiming at the image classification task, this paper proposes a game tree model to optimize the classification results, including calculating the confidence degree based on the map, the aggregation degree of the classification results, the inference value of the entities in the domain, etc., and comprehensively designing the fusion mode of knowledge graph and image classification algorithm KG-based CNN in scenarios such as multi-map input and feature pre-extraction. The mode is effective enough to enable the image classification task to utilize multimodal data, and the effectiveness is verified by multi-scenario and data ablation experiments on the public data collection.