TopicFFCA: A short-text topic-detection approach by fuzzy formal concept analysis
摘要
Over the past decade, online social networks (OSNs) like Twitter have experienced significant growth. Users generate vast amounts of short texts on these platforms. In the information era, automated topic detection is critical for managing the daily growth of social media content. However, detecting topics in short texts remains a challenging task due to their limited length, high volume, and noisy vocabulary, requiring efficient approaches to handle such complexity. This paper introduces TopicFFCA, an unsupervised technique by Fuzzy Formal Concept Analysis (FFCA) to detect topics from Twitter streams. TopicFFCA involves four steps: (1) Pre-processing the raw data, (2) Keyword extraction using KeyBert and Yake, followed by calculating word importance scores, (3) Applying Fuzzy FCA to construct hierarchical conceptual representations using a novel dependence-relevance matrix, and (4) Detecting Topics by weighted concept selection to identify representative concepts. TopicFFCA addresses uncertainty in textual data by constructing fuzzy formal concepts based on the tweet-keyword relations, filtering out irrelevant keywords, grouping meaningful terms into clusters, and processing them to eliminate useless information. To evaluate TopicFFCA, experiments focused on three aspects: a) Determine the impact of keyword extraction techniques (KeyBert vs. Yake) on our method’s performance, b) Assess how word similarity measurements (Word2vec vs. Glove vs. FastText vs. Bert) influence the model’s efficiency, and c) Benchmark TopicFFCA against the state-of-the-art methods (e.g., LDA, TopicBERT) to demonstrate its robustness in noisy short-text environments. Analyses reveal that TopicFFCA achieves the best performance with Transformers-based keyword extraction (KeyBert) and Glove embedding. Experiments on the FA Cup dataset demonstrate TopicFFCA’s superiority over compared methods. Results show 100% topic-recall at K = 6 and perfect keyword-precision at K = 2. These findings highlight its efficacy in managing the noisy nature of social media data, resolving ambiguity, and offering a robust framework for extracting meaningful topics.