Structuring Radiology Reports Using Dictionary Matching: A Comparison with ChatGPT-3.5

Efforts to leverage information technology advancements in healthcare have accelerated, with the “Medical Digital Transformation Promotion Headquarters” established to enforce legal measures for medical information use. One challenge is the diverse formats of radiology reports, complicating information integration. Increased digital reports lead to information overload for healthcare professionals. Structuring reports is a proposed solution, but it is not yet widely adopted in Japan. Our goal was to convert radiology reports into structured formats without changing radiologists’ workflows. We extracted disease names, conditions, and locations from actual CT reports using dependency parsing and dictionary matching, and compared this with structured text generated by ChatGPT-3.5. The proposed method’s accuracy averaged 61% for hospital-generated reports and 55% for ChatGPT-3.5-generated reports. In contrast, ChatGPT-3.5’s structuring accuracy averaged 92.5%, indicating its potential for practical application if ethical issues can be addressed.