<p>Clinical decision making often relies on expert judgment guided by established guidelines, which can be challenging to standardize and abstract to implement. For example, selecting between gene panels and whole exome/genome sequencing (WES/WGS) for rare disease diagnosis frequently requires interpretation of evidence-based recommendations from the American College of Medical Genetics and Genomics (ACMG) guideline. Traditional machine learning (ML) models predicting suitable genetic tests often face interpretability limitations. We hypothesize that large language models (LLMs) can be fine-tuned to “mimic” clinicians’ reasoning patterns by interpreting and applying clinical guidelines with chain-of-thought (CoT). We present RareDAI, an integrative approach that addresses this challenge by analyzing heterogeneous clinical data, including unstructured notes and structured Phecodes. Using seven domain-specific questions, we guide the Llama 3.1 and Qwen 3 models to generate structured CoT outputs. These outputs are refined via our proposed self-distillation fine-tuning (SDFT) approach, enabling the model to produce interpretable reasoning prior to recommendation. RareDAI outperforms traditional supervised fine-tuning and base LLMs (e.g., Llama 3.1, GPT-4) by up to 10–20% in all metrics (accuracy, precision, recall, and F1-score) on both in-house data and external data, effectively assisting clinicians in selecting between diagnostic modalities across healthcare systems.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Interpretable fine-tuned large language models facilitate making genetic test decisions for rare diseases

  • Quan M. Nguyen,
  • Fangyi Chen,
  • Cong Liu,
  • Ian M. Campbell,
  • Gongbo Zhang,
  • Da Wu,
  • Katherine M. Szigety,
  • Sarah E. Sheppard,
  • Priyanka Ahimaz,
  • Casey N. Ta,
  • Wendy K. Chung,
  • Chunhua Weng,
  • Kai Wang

摘要

Clinical decision making often relies on expert judgment guided by established guidelines, which can be challenging to standardize and abstract to implement. For example, selecting between gene panels and whole exome/genome sequencing (WES/WGS) for rare disease diagnosis frequently requires interpretation of evidence-based recommendations from the American College of Medical Genetics and Genomics (ACMG) guideline. Traditional machine learning (ML) models predicting suitable genetic tests often face interpretability limitations. We hypothesize that large language models (LLMs) can be fine-tuned to “mimic” clinicians’ reasoning patterns by interpreting and applying clinical guidelines with chain-of-thought (CoT). We present RareDAI, an integrative approach that addresses this challenge by analyzing heterogeneous clinical data, including unstructured notes and structured Phecodes. Using seven domain-specific questions, we guide the Llama 3.1 and Qwen 3 models to generate structured CoT outputs. These outputs are refined via our proposed self-distillation fine-tuning (SDFT) approach, enabling the model to produce interpretable reasoning prior to recommendation. RareDAI outperforms traditional supervised fine-tuning and base LLMs (e.g., Llama 3.1, GPT-4) by up to 10–20% in all metrics (accuracy, precision, recall, and F1-score) on both in-house data and external data, effectively assisting clinicians in selecting between diagnostic modalities across healthcare systems.