Contrastive Learning with Similarity Enhancement for Dimensionality Reduction

Dimensionality reduction aims to reduce the number of dimensions while preserving crucial information. As a self-supervised learning approach, contrastive learning provides a novel perspective for dimensionality reduction. However, most contrastive learning methods focus on optimizing similarity, which have limitations in reducing feature redundancy. Moreover, the dependence on negative pairs introduces computational overhead. To address these problems, we propose Contrastive Learning with Similarity Enhancement for Dimensionality Reduction (CLSDR), which integrates neighborhood embedding into the contrastive learning framework. Specifically, CLSDR uses k-nearest neighbors sampling to construct positive pairs. We design a multi-level loss function that captures the diversity of data while maintaining the consistency of local and global features. In addition, we propose a nonlinear similarity optimization mechanism based on logarithmic smoothing, which adjusts the gradient of similarity loss, improving the stability during the training process. Experimental results demonstrate that CLSDR significantly outperforms several state-of-the-art methods. Especially, on the Street View House Numbers dataset, CLSDR achieves 66.7% and 62.3% Top-1 accuracy with two classifiers in 64-dimensional embedding space, which have 10.6% and 38.4% improvement over the best competing approach. Thus, CLSDR exhibits strong robustness and scalability across different dimensions.