Federated Learning to Speed Up Pre-processing of Large Data Sets
摘要
Machine learning algorithms are data-generated algorithms, if a modest amount of historical data is available, the predictions or recommendations can be inferred easily. The learning ecosystem involves pre-processing with the standard encoding of categorical data, feature scaling, null value treatment, and a few more. The task of pre-processing is very crucial as no data is clean, but most are useful. For small data sets, the time taken for pre-processing is within some considerable limit; but for large data sets, it may significantly grow and sometimes take longer than the expected limit. With respect to the single server model, if the federation is formed among a number of available systems and the available data is distributed, the pre-processing task is likely to be accelerated. It can be achieved through parallel execution by sharing the total data among the federation peers in a judicious manner. In this work, the proposed method adopts federated learning to accelerate the pre-processing time on large data sets. The results are compared with the classical machine learning approach; which reveals that the federated learning approach outperforms the classical counterpart.