Entropy regularized fuzzy nonnegative matrix factorization for data clustering

Clustering high-dimensional data is very challenging due to the curse of dimensionality. To address this problem, low-rank matrix approximations are widely used to identify the underlying low-dimensional structure of a dataset. Among these, nonnegative matrix factorization (NMF) is the most popular because its decomposed factors are nonnegative and meaningful. However, the NMF problem has been proved to be nonconvex and NP-hard, thus resulting in many local minima. To obtain high-quality local minima, we propose an entropy regularized fuzzy nonnegative matrix factorization (ERF-NMF) model for high-dimensional data fuzzy clustering. First, probability simplex constraints on the decomposed weight components are added to achieve dimension reduction and fuzzy clustering of a dataset simultaneously. Based on the constraints, we also introduce entropy regularization to further reduce the search space for optimal solutions. Finally, we present multiplicative update rules for solving the ERF-NMF model and provide a complexity and convergence analysis. Comprehensive experiments show that the proposed ERF-NMF performs remarkably well with promising results, and its decomposition will be sparser because of entropy regularization and have a clearer physical meaning because of probability simplex constraints.