Scholarlink

Large-scale medical multimodal models are difficult to deploy in low-resource clinical environments due to high graphics memory requirements. We propose MedLite, a lightweight model that migrates zero-sample classification and cross-modality matching capabilities from BioMedCLIP via knowledge distillation, adapted to low graphics memory devices. MedLite combines MobileNetV3-Small, DistilBERT, and the single-layer multi-head attention mechanism, with 65 million total parameters and only 4GB of optimized graphics memory usage. Training is based on the ROCOv2 dataset (79,789 image-text pairs), and MIMIC-CXR (2,000 chest radiograph samples) is used for validation only. Experiments comparing BioMedCLIP, MedCLIP and MobileViT show that MedLite achieves 79.2% zero-sample classification accuracy and 82.0% recall on ROCOv2, and 75.6% and 78.4% on MIMIC-CXR, respectively, with an inference time of 55ms.MedLite strikes an excellent balance between performance and efficiency, providing an efficient multimodal analysis solution for low-resource clinical environments.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MedLite: A Lightweight Medical Multimodal Model Based on Knowledge Distillation and Inference Optimization

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Jinwei Ye,
  • Yaokang Wang,
  • Weibin Kong,
  • Gengshen Wu,
  • Wenjian Liu

摘要

Large-scale medical multimodal models are difficult to deploy in low-resource clinical environments due to high graphics memory requirements. We propose MedLite, a lightweight model that migrates zero-sample classification and cross-modality matching capabilities from BioMedCLIP via knowledge distillation, adapted to low graphics memory devices. MedLite combines MobileNetV3-Small, DistilBERT, and the single-layer multi-head attention mechanism, with 65 million total parameters and only 4GB of optimized graphics memory usage. Training is based on the ROCOv2 dataset (79,789 image-text pairs), and MIMIC-CXR (2,000 chest radiograph samples) is used for validation only. Experiments comparing BioMedCLIP, MedCLIP and MobileViT show that MedLite achieves 79.2% zero-sample classification accuracy and 82.0% recall on ROCOv2, and 75.6% and 78.4% on MIMIC-CXR, respectively, with an inference time of 55ms.MedLite strikes an excellent balance between performance and efficiency, providing an efficient multimodal analysis solution for low-resource clinical environments.