A Domain-Specific Information Based Multi-Domain Combinatorial Ensemble Method for Multi-Source Domain Generalization

In the multi-source domain training scenario, due to the fact that a single universal model trained by the existing domain generalization methods may have a large adaptive gap with the optimal classification model of the target domain, we propose a multi-domain combination ensemble generalization method to expand the hypothesis space of the classification model and stabilize the model performance. First, new training domains with different data distributions are constructed into multi-domain training data, and a network model with multiple classifier heads is constructed at the same time. Second, the feature extractor of this model is trained on the entire training data, while different classifier heads are trained on different training domains. Then, during the training process, the overall weight parameters of the model are smoothed by the method of weight averaging to stabilize the model performance. Finally, the smoothed parameters are used for ensemble prediction of multiple classifier heads through entropy minimization. Experiment results show that our multi-source domain generalization method can effectively improve the out-ofdistribution generalization performance of the classification model.