Location-Guided Automated Lesion Captioning in Whole-Body PET/CT Images
摘要
Whole-body PET/CT imaging provides detailed metabolic and anatomical information, which is critical for accurate cancer staging, treatment evaluation, and radiotherapy planning. Automated lesion captioning for whole-body PET/CT is essential for reducing radiologists’ workload and assisting personalized treatment decisions. Unlike previous works that focus on captioning body-part images, we propose a novel automated lesion captioning framework for whole-body PET/CT images, which usually have large volume and high anatomical variability. Our framework first leverages CLIP for lesion localization, upon which we introduce two location-guided strategies: Confidence-Guided Location Prompts (CGLP), which select top-1 or top-3 anatomical location prompts based on confidence scores to guide captioning, and Dynamic Window Setting (DWS), which applies appropriate intensity windowing to enhance visual representation of the localized regions. To our knowledge, our work is the first to achieve whole-body PET/CT lesion captioning. Experimental results on a large dataset comprising 1867 subjects from Siemens, GE, and United Imaging show that our method not only yields higher BLEU scores compared to state-of-the-art methods, but also produces consistent improvements across multiple scanner makers. This advancement has the potential to streamline radiology reporting and enhance clinical decision-making using whole-body PET/CT images.