Distributed extreme learning machine with alternating direction method of multiplier

Extreme learning machine, as a generalized single-hidden-layer feedforward network, has achieved much attention for its extremely fast learning speed and good generalization performance. However, big data often makes a challenge in large scale learning of extreme learning machine due to the memory limitation of single machine as well as the distributed manner of large scale data in many applications. For the purpose of relieving the limitation of memory with big data, in this paper, we exploit a novel distributed model to implement the extreme learning machine algorithm in parallel for large-scale data set, namely distributed extreme learning machine (DELM). A corresponding algorithm is developed on the basis of alternating direction method of multipliers which has shown its effectiveness in distributed convex optimization. Finally, extensive experiments on some benchmark data sets are carried out to illustrate the effectiveness and superiority of the proposed DELM method with an analysis on the performance of speedup, scaleup and sizeup.