TY - JOUR
T1 - Geometry-Stable Class-Incremental Semantic Segmentation
AU - Song, Xiang
AU - Yang, Ziye
AU - He, Yuhang
AU - Chen, Jie
AU - Comminiello, Danilo
AU - Gong, Yihong
N1 - Publisher Copyright:
© 1991-2012 IEEE.
PY - 2026
Y1 - 2026
N2 - Class-Incremental Semantic Segmentation (CISS) requires a model to continually learn to segment new classes without forgetting previously learned ones. Existing CISS methods attempt to preserve old knowledge by replaying or mimicking past predictions. However, they fail to stabilize the underlying feature geometry under shifting background semantics, leading to prototype drift and forgetting. To address this problem, we propose a geometry-stable CISS paradigm that imposes an Orthogonal Equiangular Tight Frame (O-ETF) classifier. Our O-ETF keeps all foreground prototypes on a simplex ETF while assigning the background a dedicated prototype strictly orthogonal to the foreground span. This decouples background supervision from foreground semantics, providing geometry-guided supervision that helps mitigate foreground-background interference under background semantic shift. Based on this geometry, we build a shared-encoder and task-specific-decoder framework, where lightweight LoRA modules are used to adapt each decoder branch to newly introduced classes. During inference, we further design a Bayesian product-of-experts (Bayes-PoE) fusion strategy, which lifts branch-level predictions to the accumulated global label space and aggregates them in the shared O-ETF semantic space. Extensive experiments on ADE20K and Pascal VOC 2012 demonstrate the effectiveness of our method.
AB - Class-Incremental Semantic Segmentation (CISS) requires a model to continually learn to segment new classes without forgetting previously learned ones. Existing CISS methods attempt to preserve old knowledge by replaying or mimicking past predictions. However, they fail to stabilize the underlying feature geometry under shifting background semantics, leading to prototype drift and forgetting. To address this problem, we propose a geometry-stable CISS paradigm that imposes an Orthogonal Equiangular Tight Frame (O-ETF) classifier. Our O-ETF keeps all foreground prototypes on a simplex ETF while assigning the background a dedicated prototype strictly orthogonal to the foreground span. This decouples background supervision from foreground semantics, providing geometry-guided supervision that helps mitigate foreground-background interference under background semantic shift. Based on this geometry, we build a shared-encoder and task-specific-decoder framework, where lightweight LoRA modules are used to adapt each decoder branch to newly introduced classes. During inference, we further design a Bayesian product-of-experts (Bayes-PoE) fusion strategy, which lifts branch-level predictions to the accumulated global label space and aggregates them in the shared O-ETF semantic space. Extensive experiments on ADE20K and Pascal VOC 2012 demonstrate the effectiveness of our method.
KW - Class-incremental learning
KW - Geometric stability
KW - Neural collapse theory
KW - Semantic segmentation
UR - https://www.scopus.com/pages/publications/105054030413
U2 - 10.1109/TCSVT.2026.3743780
DO - 10.1109/TCSVT.2026.3743780
M3 - 文章
AN - SCOPUS:105054030413
SN - 1051-8215
JO - IEEE Transactions on Circuits and Systems for Video Technology
JF - IEEE Transactions on Circuits and Systems for Video Technology
ER -