Dimension reduction based on a penalized kernel support vector machine model

Prediction and dimension reduction play increasingly significant roles in high-dimensional data analysis; however, they suffer from model inaccuracy, selection inconsistency and a prohibitively expensive computational cost as the model dimension increases exponentially. Although a support vector machine (SVM) is one of the most powerful forecasting approaches that are widely used by the research community, it does not provide an interpretable model that violates the principle of Occam's razor. In classical regression problems, a penalized linear SVM model for the dimension reduction task has been investigated in the linear feature space under the assumption that the underlying true model is linear. In this paper, the penalized kernel SVM (PKSVM) model is proposed and investigated combining with a SVM information criterion (SVMIC) for the dimension reduction task in the nonlinear kernel space using radial basis function for prediction and model representation. Instead of pursuing sparsity in the original feature space of the SVM, dimension reduction is performed in the kernel feature space. Computationally, a fast and simple-to-implement algorithm is derived. Furthermore, both SVMIC and cross-validation are utilized to select the kernel and regularization parameters to guarantee the model consistency. Real data applications including microarray data analysis and global solar radiation forecasting are provided. The first application selected significant components in genetic study and the important data information is extracted for grid-connected photovoltaic installations in the second application. The proposed dimension reduction technique avoids the model redundancy which is of great importance for knowledge discovery.