Scholarlink

Background <p>AUA guidelines for shared decision making (SDM) in prostate cancer recommend discussion of five content areas in consultations: (1) cancer severity (tumor risk (TR), pathology results (PR)); (2) life expectancy (LE); (3) cancer prognosis (CP); (4) baseline urinary and erectile function (UF and EF); and (5) treatment side effects (erectile dysfunction (ED), urinary incontinence (UI), and irritative urinary symptoms (LUTS)). However, patient retention of information after the visit and inconsistent risk communication by physicians are barriers to informed SDM. We sought to develop natural language processing (NLP) models based on recorded consultations to provide key information to patients and audit quality of physician communication.</p> Methods <p>We used 50 consultation transcripts to train and validate NLP models to identify sentences related to key concepts. We then tested whether communication quality across entire consultations could be determined by sentences with the highest model-predicted topic concordance in 20 separate consultation transcripts.</p> Results <p>Our development dataset included 28,927 total sentences, with 75% reserved for training and 25% for internal validation. The Random Forest model had the highest accuracy for identifying topic-concordant sentences, with area under the curve 0.98, 0.94, 0.89, 0.92, 0.84, 0.96, 0.98, 0.97, and 0.99 for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding across all concepts in the internal validation dataset. In 20 separate consultations, the top 10 model-identified sentences correctly graded communication quality across entire consultations with accuracies of 100%, 90%, 95%, 95%, 80%, 95%, 85%, 100%, and 95% for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding, respectively.</p> Conclusions <p>NLP models accurately capture key information and grade quality of physician communication in prostate cancer consultations, providing the foundation for scalable quality assessment of risk communication.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Development and validation of a natural language processing system to assess quality of physician communication in prostate cancer consultations

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年8月21日

查看全文
  • Renning Zheng,
  • Nadine A. Friedrich,
  • Michael Luu,
  • Rebecca Gale,
  • Dmitry Khodyakov,
  • Stephen J. Freedland,
  • Brennan Spiegel,
  • Timothy J. Daskivich

摘要

Background

AUA guidelines for shared decision making (SDM) in prostate cancer recommend discussion of five content areas in consultations: (1) cancer severity (tumor risk (TR), pathology results (PR)); (2) life expectancy (LE); (3) cancer prognosis (CP); (4) baseline urinary and erectile function (UF and EF); and (5) treatment side effects (erectile dysfunction (ED), urinary incontinence (UI), and irritative urinary symptoms (LUTS)). However, patient retention of information after the visit and inconsistent risk communication by physicians are barriers to informed SDM. We sought to develop natural language processing (NLP) models based on recorded consultations to provide key information to patients and audit quality of physician communication.

Methods

We used 50 consultation transcripts to train and validate NLP models to identify sentences related to key concepts. We then tested whether communication quality across entire consultations could be determined by sentences with the highest model-predicted topic concordance in 20 separate consultation transcripts.

Results

Our development dataset included 28,927 total sentences, with 75% reserved for training and 25% for internal validation. The Random Forest model had the highest accuracy for identifying topic-concordant sentences, with area under the curve 0.98, 0.94, 0.89, 0.92, 0.84, 0.96, 0.98, 0.97, and 0.99 for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding across all concepts in the internal validation dataset. In 20 separate consultations, the top 10 model-identified sentences correctly graded communication quality across entire consultations with accuracies of 100%, 90%, 95%, 95%, 80%, 95%, 85%, 100%, and 95% for TR, PR, LE, CP, UF, EF, ED, UI, and LUTS compared with manual coding, respectively.

Conclusions

NLP models accurately capture key information and grade quality of physician communication in prostate cancer consultations, providing the foundation for scalable quality assessment of risk communication.