Sector-Based Incremental Clustering and Scalable Deletion for Real-Time Big Data Streaming Application

Massive streaming data is the most current technology for storing and manipulating large quantities of data. Processing the substantial amount of streaming data is still a challenging issue. The speed and throughput can be improved while extracting this huge amount of data by using a dynamic query processing technique. The sector-based incremental clustering indexing and scalable deletion approach is used in this paper to improve the system performance. Stream processing can be more effective and flexible by applying dynamic query processing technique. Incremental clustering using sector-based method and scalable deletion which removes unwanted, spurious data is proposed in this work in order to process incoming streaming data effectively. Incremental clustering is one of the clustering methods used for processing dynamic information. It is not necessary to plan the entire dataset ahead of time in this method which is suitable for live streaming data, because it processes one instance at a time. So, less space and time is used. Various queries with different types of datasets executed using sector based incremental clustering and scalable deletion methods. The proposed system achieves the highest results for more than 90% accuracy with less space and more speed when the execution time and data retrieval are empirically increased compared with conventional approaches.