Topic modeling and Natural Language Processing (NLP) have demonstrated significant prospects in the healthcare industry for extracting insightful information from unstructured patient histories that can help diagnose diseases and enhance clinical decisions. In this study, patient histories are grouped into ten different clusters using advanced K-Means clustering, with the Dunn Index being used to validate the clustering performance. After the clusters are formed, each cluster is subjected to topic modeling approaches. Four topic modeling approaches are examined in this study, Latent Dirichlet Allocation (LDA), Hierarchical Dirichlet Process (HDP), Latent Semantic Indexing (LSI), and Non-negative Matrix Factorization (NMF). These techniques are used to find disease-related terms from patient histories. Coherence scores, which show the semantic significance of the terms produced, and execution times, which show the computational efficiency needed for real-time healthcare applications, are used to evaluate the models. According to experimental findings for the USMLE® Step 2 Clinical Skills exam dataset, NMF and HDP generated the most cohesive terms, with NMF’s faster execution time (1.67 s) making it appropriate for widespread healthcare applications. Whereas, a reasonable balance between coherence and computational demands is offered by LDA and LSI.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

NLP and Topic Modeling in Healthcare: Identifying Diseases from Patient Histories

  • Sujatha Arun Kokatnoor,
  • Samiksha Shukla

摘要

Topic modeling and Natural Language Processing (NLP) have demonstrated significant prospects in the healthcare industry for extracting insightful information from unstructured patient histories that can help diagnose diseases and enhance clinical decisions. In this study, patient histories are grouped into ten different clusters using advanced K-Means clustering, with the Dunn Index being used to validate the clustering performance. After the clusters are formed, each cluster is subjected to topic modeling approaches. Four topic modeling approaches are examined in this study, Latent Dirichlet Allocation (LDA), Hierarchical Dirichlet Process (HDP), Latent Semantic Indexing (LSI), and Non-negative Matrix Factorization (NMF). These techniques are used to find disease-related terms from patient histories. Coherence scores, which show the semantic significance of the terms produced, and execution times, which show the computational efficiency needed for real-time healthcare applications, are used to evaluate the models. According to experimental findings for the USMLE® Step 2 Clinical Skills exam dataset, NMF and HDP generated the most cohesive terms, with NMF’s faster execution time (1.67 s) making it appropriate for widespread healthcare applications. Whereas, a reasonable balance between coherence and computational demands is offered by LDA and LSI.