Enhancing Data Flow Architecture and Cleaning Effectiveness in Industry 4.0: A Case Study of Data Cleaning Using DBSCAN

Industry 4.0 endeavors to elevate productivity by efficiently gathering and processing real-time data. It relies on the Internet of Things (IoT) for data capture and harnesses the scalable computing resources within the Big Data ecosystem to manage the extensive and diverse data generated by various sensors. Moreover, Industry 4.0 integrates IoT analytics and Machine Learning techniques to translate data into actionable insights. Despite these advancements, the lack of research on open-source data flow platforms underscores the pressing need to address these requirements. In this paper, we propose a comprehensive architecture for the acquisition and processing of sensor data, incorporating a local pre-processing layer named “Edge computing”. Additionally, we delve into various anomaly detection approaches to facilitate data cleansing, with a specific focus on evaluating the efficacy of DBSCAN, a widely utilized clustering algorithm in this specific context.