NLP and Topic Modeling in Healthcare: Identifying Diseases from Patient Histories
摘要
Topic modeling and Natural Language Processing (NLP) have demonstrated significant prospects in the healthcare industry for extracting insightful information from unstructured patient histories that can help diagnose diseases and enhance clinical decisions. In this study, patient histories are grouped into ten different clusters using advanced K-Means clustering, with the Dunn Index being used to validate the clustering performance. After the clusters are formed, each cluster is subjected to topic modeling approaches. Four topic modeling approaches are examined in this study, Latent Dirichlet Allocation (LDA), Hierarchical Dirichlet Process (HDP), Latent Semantic Indexing (LSI), and Non-negative Matrix Factorization (NMF). These techniques are used to find disease-related terms from patient histories. Coherence scores, which show the semantic significance of the terms produced, and execution times, which show the computational efficiency needed for real-time healthcare applications, are used to evaluate the models. According to experimental findings for the USMLE® Step 2 Clinical Skills exam dataset, NMF and HDP generated the most cohesive terms, with NMF’s faster execution time (1.67 s) making it appropriate for widespread healthcare applications. Whereas, a reasonable balance between coherence and computational demands is offered by LDA and LSI.