Topic Modeling for Enhancing Transformers Hate Speech Detection
摘要
This paper proposes TFusion, a novel text classification fra-mework that integrates topic modeling from Latent Dirichlet Allocation (LDA) and deep contextual embeddings from Large Language Models such as BERT. LDA learns topic representations of samples, capturing word-frequency dependent features, while the transformer generates deep contextual embeddings, capturing context-based features. Model-level fusion is used to combine these complementary sets of features to enhance predictive performance. This paper addresses the topic of Hate Speech detection, a specific field of Natural Language Processing of high academic, governmental and corporate interest over the last decade, and applies TFusion to this problem. The framework was tested on the Stormfront Hate Speech Dataset, chosen for being one the most challenging in the field, in which state-of-the-art approaches achieve some of their lowest performances. The carried experiments used a Distil-BERT encoder inside TFusion due to its state-of-the-art performance in Hate Speech detection coupled with a lower computational demand. Moreover, 10-fold validation was conducted 10 times, totaling 100 tests, to validate the obtained results. The results show that the proposed framework outperforms both LDA coupled with a classifier and Distil-BERT, in terms of macro F1 score in the hate speech detection task (p-value=0.0046).