Scholarlink

<p>With the continuous advancement of artificial intelligence technology, Medical Visual Question Answering (Med-VQA) systems have demonstrated significant potential in the field of medical diagnosis and treatment. However, existing Med-VQA models often overlook the importance of medical expertise, resulting in inadequate accuracy and adaptability in complex scenarios. Even when external knowledge bases are introduced, they do not effectively utilize medical knowledge and images to enhance semantic understanding and knowledge reasoning related to multimodal information. Inspired by the theory of internalization in cognitive psychology, this paper proposes an Image-Knowledge Internalization Assistance (IKIA) model for Medical Visual Question Answering, aimed at enhancing the semantic understanding and reasoning abilities of Med-VQA models by integrating medical images, texts, and knowledge. The framework consists of two main modules: the Image-Knowledge Internalized Information Generator and the Image-Knowledge Internalization Assistance Module. First, unimodal encoders are utilized to extract features from medical images and knowledge base texts separately. Subsequently, the image-knowledge multimodal fusion internalization operation generates the image-knowledge internalized information. Finally, the image-knowledge internalized information is injected into the Med-VQA model to assist in improving the model’s accuracy and its ability to handle complex Med-VQA tasks. Experimental results show that this method achieves significant performance improvements on the VQA-RAD and SLAKE datasets, demonstrating its effectiveness and superiority.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

IKIA: Image-Knowledge Internalization Assistance Model for Medical Visual Question Answering

  • 期刊论文

  • 发表日期：

    2026年2月16日

查看全文
  • Yurun Bi,
  • Xingang Wang,
  • Yuteng Xiao,
  • Yudong Zhang

摘要

With the continuous advancement of artificial intelligence technology, Medical Visual Question Answering (Med-VQA) systems have demonstrated significant potential in the field of medical diagnosis and treatment. However, existing Med-VQA models often overlook the importance of medical expertise, resulting in inadequate accuracy and adaptability in complex scenarios. Even when external knowledge bases are introduced, they do not effectively utilize medical knowledge and images to enhance semantic understanding and knowledge reasoning related to multimodal information. Inspired by the theory of internalization in cognitive psychology, this paper proposes an Image-Knowledge Internalization Assistance (IKIA) model for Medical Visual Question Answering, aimed at enhancing the semantic understanding and reasoning abilities of Med-VQA models by integrating medical images, texts, and knowledge. The framework consists of two main modules: the Image-Knowledge Internalized Information Generator and the Image-Knowledge Internalization Assistance Module. First, unimodal encoders are utilized to extract features from medical images and knowledge base texts separately. Subsequently, the image-knowledge multimodal fusion internalization operation generates the image-knowledge internalized information. Finally, the image-knowledge internalized information is injected into the Med-VQA model to assist in improving the model’s accuracy and its ability to handle complex Med-VQA tasks. Experimental results show that this method achieves significant performance improvements on the VQA-RAD and SLAKE datasets, demonstrating its effectiveness and superiority.