Diagnosing BN discrepancy in rehearsal-based class incremental learning

To consolidate previous knowledge, various rehearsal strategies have been proposed in the existing class incremental learning (CIL) methods. Albeit achieving promising performance, the underlying rationality has not been fully discussed. In this study, motivated by the potential mismatch of batch normalization (BN) statistics between training and inference phases, we analyze the BN technique adopted in these rehearsal strategies. Specifically, through exploratory experiments and detailed analysis, we investigate the BN discrepancy issue and elaborate its underlying causes and influences. We find that although such BN discrepancy may degrade testing performance, it is potential to help alleviate the ubiquitous class imbalance problem in rehearsal-based CIL models. Inspired by such observations, we propose a novel rehearsal strategy named Split with sTabilized EMA Statistics (STEMS), to regulate the updating of BN statistics and reduce the classification bias for rehearsal-based CIL methods. Through comprehensive experiments conducted on various benchmark CIL datasets with disjoint and blurry task boundaries, we show that STEMS can bring significant performance gains to various rehearsal-based CIL methods, revealing its potential generality along this line of research.