LSiX: A Scheme for Efficient Multiple Continuous Window Aggregation Over Streams

Stream processing engines need to process multiple queries over streams simultaneously, and continuous window aggregation plays a critical role in various applications as a part of data analysis pipelines. However, the system suffers from scalability issues when dealing with massive queries with different window and slide sizes over data streams with high input rates. To address this problem, we propose LSiX (longest-shortest-window-based indexing) to aggregate multiple queries over data streams efficiently. Specifically, we use two arrays based on the longest and shortest windows among all registered queries, and all query results are computed by sharing partial aggregation from the two arrays. This allows for efficient aggregate computation, with a maximum of two operations for each query. The results of our experiment show that LSiX can be at least 3 times faster than the comparative methods, including the state-of-the-art method, MCQA.