Knowledge-Driven Hypothesis Generation for Burn Diagnosis from Ultrasound with Vision-Language Model
摘要
Although vision-language models (VLMs) have achieved strong results in general computer vision tasks, their effectiveness in medical imaging remains limited—primarily due to their insufficient reasoning capabilities. In this work, we introduce KODER, a novel knowledge-driven reasoning framework aimed at improving diagnostic accuracy for ultrasound-based burn assessment. KODER integrates pre-trained VLMs with first-order logic (FOL) reasoning to generate interpretable diagnostic hypotheses. By combining rich experimental descriptions and clinical insights into a unified prompt, the framework produces multiple diagnostic hypotheses and refines them through iterative consistency checks using an SMT solver. The validated hypotheses are then used to support both surgical decision-making and detailed burn depth classification. We evaluate our approach on a retrospective dataset collected from a U.S. burn center, where it achieves significant performance gains—reaching up to 93% accuracy in surgical classification and 87% in fine-grained burn depth prediction. Additionally, incorporating techniques such as chain-of-thought reasoning, self-consistency, and explicit explanation generation further boosts both interpretability and diagnostic reliability. Our experiments span multiple state-of-the-art VLMs, including GPT-4o, GPT-4 Turbo, and Gemini 1.5 and Gemini 2.0, confirming the generalizability of KODER across architectures.