Investigating the effect of randomly selected feature subsets on bagging and boosting

Bagging, boosting, and random subspace methods are three most commonly used approaches for constructing ensemble classifiers. In this article, the effect of randomly selected feature subsets (intersectant or disjoint) on bagging and boosting is investigated. The performance of the related ensemble methods are compared by conducting experiments on some UCI benchmark datasets. The results demonstrate that bagging can be generally improved using the randomly selected feature subsets whereas boosting can only be optimized in some cases. Furthermore, the diversity between classifiers in an ensemble is also discussed and related to the prediction accuracy of the ensemble classifier.