Named Entity Recognition (NER) in biomedical research articles is an essential task for the automation of data extraction and enabling downstream clinical analysis. In this work, a pre-existing Vision-Language Model (Llama 11B-Vision) on the annotated dataset is used to improve its ability to recognize and extract named entities from medical research paper images accurately. A high-capacity model, Llama 90B-Vision is used to generate NER annotations that serve as ground truth value. The baseline performance of the smaller model, Llama 11B-Vision is compared to this ground truth values using typical precision, recall, and F1 score metrics for its NER predictions. This smaller model is next fine-tuned on a custom dataset developed with ideal annotations to enhance on its domain-specific performance. This paper focuses on data preprocessing, model testing, and driver integration for automated NER extraction. Experimental results show substantial improvements with respect to precision after fine-tuning, substantiating the effectiveness of the suggested approach to NER for medical research papers. Results obtained shows precision of 80% for the fine-tuned Llama 11B model over Llama 11B model without fine-tuning, which achieves a precision of only 72%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fine-Tuning Strategies for Named Entity Recognition in Medical Text Images Using VLMs

  • Petluri Sai Dhruv,
  • S. Niranjan,
  • S. Varun,
  • Rushi Mayur,
  • Rajashree Shettar

摘要

Named Entity Recognition (NER) in biomedical research articles is an essential task for the automation of data extraction and enabling downstream clinical analysis. In this work, a pre-existing Vision-Language Model (Llama 11B-Vision) on the annotated dataset is used to improve its ability to recognize and extract named entities from medical research paper images accurately. A high-capacity model, Llama 90B-Vision is used to generate NER annotations that serve as ground truth value. The baseline performance of the smaller model, Llama 11B-Vision is compared to this ground truth values using typical precision, recall, and F1 score metrics for its NER predictions. This smaller model is next fine-tuned on a custom dataset developed with ideal annotations to enhance on its domain-specific performance. This paper focuses on data preprocessing, model testing, and driver integration for automated NER extraction. Experimental results show substantial improvements with respect to precision after fine-tuning, substantiating the effectiveness of the suggested approach to NER for medical research papers. Results obtained shows precision of 80% for the fine-tuned Llama 11B model over Llama 11B model without fine-tuning, which achieves a precision of only 72%.