Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications

Ensemble learning methods, such as boosting, focus on producing a strong classifier based on numerous weak classifiers. In this paper, we develop a novel ensemble learning method called rescaled boosting with truncation (ReBooT) for binary classification by combining well-known rescaling and regularization ideas in boosting. Theoretically, we present some sufficient conditions for the convergence of ReBooT, derive an almost optimal numerical convergence rate, and deduce fast-learning rates in the framework of statistical learning theory. Experimentally, we conduct both toy simulations and four real-world data runs to show the power of ReBooT. Our results show that, compared with the existing boosting algorithms, ReBooT possesses better learning performance and interpretability in terms of solid theoretical guarantees, perfect structure constraints, and good prediction performance.