A Comparative Study of Methods for Topic Modelling in News Articles

The past few decades have seen an increase in textual data and influence from the news media. With the rise in available data, especially in regard to textual data from news media, it is imperative to quickly categorise news topics. In this research, the primary aim is to suggest a method for automatically identifying news topics in articles. The dataset used in this research was the news category published on Kaggle and comprised of 210,294 headlines and abstracts from HuffPost between 2012 and 2022. The dataset consisted of a total of 42 categories and six columns. Traditional modelling techniques did not perform well in comparison with Top2Vec, NMF or BERTopic. This research confirms the efficacy of Top2Vec and BERTopic, followed by NMF, LDA and LSA for analysing, news category data from a human-interpretation perspective. Though BERTopic was able to deduce 1145 topics from the data, it could not chuck unwanted words like “to”, “say”, “for” which do not add any value to the topic semantics. In summary, TF-IDF proved to be the best feature extraction technique and Top2Vec the best topic modelling technique.