Distributed Memory-Efficient Algorithm for Extreme Learning Machines Based on Spark

This work presents a distributed limited-memory algorithm for Extreme Learning Machines (ELM) with training data stored in Spark. The method runs batch matrix computations to obtain the direct non-iterative solution of ELM, reads data only once, and uses the least network bandwidth, making it very computationally efficient. This is achieved by extensive use of lazy evaluation and generators in Spark, deliberately avoiding operations that may lead to data caching. The method scales to virtually infinite datasets and any number of ELM neurons, with runtime being the only restricting factor. An experiment demonstrates the successful processing of 1 TB of text data on an average desktop in 6.5 h without using disk cache or running out of memory. The experimental code is linked in GitHub at https://github.com/akusok/pyspark-elm .