Cin-Seg: Causal Invariance for Tag-Supervised Segmentation on Medical Images

Weakly supervised semantic segmentation (WSSS) is the method of learning a segmentation model with only weak labels, e.g., image-level labels. For WSSS methods, the segmentation result cannot be directly learned because of the lack of pixel-level annotation. Existing methods use category-highly-correlated region to approximate pixel-level segmentation result of target object. However, the category-highly-correlated region has no causal relation with the segmentation result. In this article, we propose a causal-invariance-based method to learn the intrinsic attribute that is causally related to the segmentation result of the target object. We model WSSS task as a causal problem from the perspective of causal invariance based on the fact that the intrinsic attribute of the target object is invariant. According to the causal modeling, it is feasible to learn the intrinsic attributes of target object without the supervision of pixel-level annotation. We propose a causal-invariant transformation (CIT) strategy to explicitly supervise the model to learn the causal feature of segmentation by using a multibranch architecture. The CIT constrains the prediction to follow the intrinsic attribute of target object, which improves the accuracy of segmentation result with only tag supervision. Experimental results on three medical datasets show that the proposed method outperforms state-of-the-art WSSS methods. The code is available at https://github.com/cz-xjtu/Cin-Seg.