Relative gradient speeding up additive updates for nonnegative matrix factorization

There exist two kinds of iterative updates for nonnegative matrix factorization: additive and multiplicative. The former does not take into consideration the characteristic of the parameter space of the constrained optimization while the latter holds the nonnegativity well. The relative gradient has better convergence rate than the ordinary gradient, and has been successfully used for neural learning, especially for blind source separation and independent component analysis. This paper applies the relative gradient to speed up the additive updates for nonnegative matrix factorization according to square Euclidean error. The primary experiments on synthetic and real datasets demonstrate the effectiveness of the proposed method.