A regression algorithm based on AdaBoost

The key to successful machine learning methods is learning quality or accuracy. Boosting Is has proved to be effective method for improving learning quality of a weak learning algorithm with wide applications to classifications problems. However few successful applications were reported on improving regression quality by boosting. This paper presents a new algorithm for regression based on a boosted support vector machine (SVM) method. A regression problem is first converted to a binary classification problem upon the concept of ε-insensitive loss. By applying the idea of AdaBoost algorithm, an optimal classification-plane ensemble is constructed with the converted classification data set. Based on this ensemble a regression estimate function is obtained with equivalence to the original regression problem. The analysis shows that for the regression data set, the number of samples with regression error exceeding e will decreased exponentially with the number of Boosting iterations. The testing results for an actual data set show that the new algorithm is effective.