The exponential growth of scientific literature necessitates advanced computational approaches for effective text analysis and classification. This study investigates the enhancement of Latent Dirichlet Allocation (LDA) through integration with semantic vector representations—Word2Vec, Doc2Vec, and SciBERT—for improved clustering of academic abstracts. We systematically compare the performance of three hybrid models: LDA + Word2Vec, LDA + Doc2Vec, and LDA + SciBERT, using multiple evaluation metrics including topic coherence, entropy, and clustering indices (Davies-Bouldin Index, Silhouette Score, Calinski-Harabasz Index). Results demonstrate that LDA combined with Word2Vec achieves superior clustering stability and topic coherence, especially for small-scale datasets. While Doc2Vec enhances document-level semantics, it exhibits variability in clustering performance across different topic configurations. SciBERT, although capable of capturing deep contextual meaning, faces challenges when integrated directly with LDA due to representation mismatches. These findings offer practical guidance for selecting optimal embedding-based topic modeling strategies in scientific literature analysis. The study contributes to the growing field of hybrid NLP models by identifying strengths and limitations of each approach, and suggests future work involving hierarchical clustering and neural-topic modeling to improve thematic separation and interpretability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Topic Modeling in Scientific Literature: A Comparative Study of LDA with Word2Vec, Doc2Vec, and SciBERT Embeddings

  • Roqyia Abdelhamed,
  • Mansour Essgaer,
  • Asma Agaal,
  • Abubaker Shibani

摘要

The exponential growth of scientific literature necessitates advanced computational approaches for effective text analysis and classification. This study investigates the enhancement of Latent Dirichlet Allocation (LDA) through integration with semantic vector representations—Word2Vec, Doc2Vec, and SciBERT—for improved clustering of academic abstracts. We systematically compare the performance of three hybrid models: LDA + Word2Vec, LDA + Doc2Vec, and LDA + SciBERT, using multiple evaluation metrics including topic coherence, entropy, and clustering indices (Davies-Bouldin Index, Silhouette Score, Calinski-Harabasz Index). Results demonstrate that LDA combined with Word2Vec achieves superior clustering stability and topic coherence, especially for small-scale datasets. While Doc2Vec enhances document-level semantics, it exhibits variability in clustering performance across different topic configurations. SciBERT, although capable of capturing deep contextual meaning, faces challenges when integrated directly with LDA due to representation mismatches. These findings offer practical guidance for selecting optimal embedding-based topic modeling strategies in scientific literature analysis. The study contributes to the growing field of hybrid NLP models by identifying strengths and limitations of each approach, and suggests future work involving hierarchical clustering and neural-topic modeling to improve thematic separation and interpretability.