Optimized Latent-Dirichlet-Allocation Based Topic Modeling–An Empirical Study

Topic modeling is an unsupervised learning based mechanism used to uncover hidden topics from the voluminous corpus generated out of social media, environment, medicine and all other viable domain. It can be accomplished using a variety of topic modeling techniques based on the types of data, such as complex and short text data. In this article an improved version of LDA named as Optimized LDA (OLDA) is proposed using hyperparameter tuning in order to extract the hidden topics from the huge volume of corpus. The OLDA was trained and tested on three datasets; viz.., Newsgroup, Top Reddit Data and Upvote Data. The performance of OLDA was analyzed and validated by comparing with the existing standard methodologies namely Latent Semantic Analysis (LSA) and Probabilistic Latent Semantic Analysis (PLSA). The improved coherence score of OLDA confirms on its merits and potential in accurate and contextual topic modeling.