Compact ensemble model of neural networks

Analyzing the construction process of the existing neural network ensembles, a novel compact ensemble model of neural networks is proposed. Training members in an ensemble and optimizing the combination weights for members are carried out simultaneously in the same learning process, and all parameters are adjusted to improve the generalization ability of the ensemble. In comparison with other existing ensemble model, the ensemble construction process is more compact by integrating the two stages into one, and the communication among neural networks is based on the real-time and dynamic structure of an ensemble so that the information conveyed between integration and training always keeps coincident. To validate the validity and advantage of this ensemble model, four well known classification problems are considered to compare the generalization error of compact ensemble with the generalization error of CNNE, Bagging, Boosting and other existing neural network ensemble model. The experimental results show that the error rate on testing sets can be decreased with compact ensemble model by 8% to 16%.