Detection and context reconstruction of sub-events that influence the course of a news event from microblog discussions

Social media has become an inevitable part of human communication and the primary source for reading and tracking news events. Most news events constitute minor independent events related to the same news event, referred to as sub-events. Sub-events add more layers and dimensions to the analysis of events, making them vital during emergencies and political movements. The task of sub-event detection from microblogs aims to provide a concise summary of the sub-events being discussed. Sub-event detection is a complex problem, primarily due to the vast mass of data and overlapping vocabulary. Despite the challenges, this task is crucial for government policymakers to help them understand the pulse of the common public and make informed decisions. The proposed work detects sub-events as word clusters using a novel non-probabilistic, incremental clustering method based on the correlation patterns of microblog text. Initially, documents of co-occurrence are generated for each word in the dataset above a minimal threshold frequency. Similar co-occurrence documents are grouped. The documents are compared using a novel similarity measure, and clusters of keywords are generated as candidates. The candidates are validated, processed and merged based on a set of rules to generate sub-event clusters. This work also proposes a primitive context induction process that provides a high-level understanding of the sub-event without requiring prior domain knowledge of the news articles. The significance of this method lies in its ability to unearth sub-events along with their initial context without requiring pre-annotated ground truth, background information, or domain-specific training. This work validates the identified sub-events by checking against actual news articles discussing the sub-event. The proposed method proves highly effective in capturing both newly emerging and ongoing sub-events in the news events. Additionally, it exhibits a strong ability to recognise minor sub-events. The performance of this work is evaluated by comparing it against the topic modelling algorithm and K-means clustering algorithm and proves to be efficient.