Efficient Mining of Association Rules Using Apache Spark
摘要
The exponential growth of data in the era of big data and the Internet of Things (IoT) has made traditional centralized association rule mining algorithms inefficient. These algorithms encounter several challenges, including bandwidth limitations, energy constraints, low processing power, privacy concerns, insufficient storage capacity, and limited scalability. This led to the development of Distributed Association Rule Mining (D-ARM). This study implements the FP-Growth algorithm in a parallel computing environment using Apache Spark. The proposed approach enhances scalability and computational efficiency by leveraging Spark’s distributed processing capabilities. The performance of the distributed FP-Growth algorithm is evaluated and compared with the traditional centralized approach using different datasets. Experimental results demonstrate that the Spark-based implementation significantly reduces execution time and improves efficiency, making it suitable for large-scale data mining tasks. This study highlights the advantages of distributed computing in mining association rules and provides insights into the optimization of parallel data mining techniques.