Exploring few-shot defect segmentation in general industrial scenarios with metric learning and vision foundation models

Industrial defect segmentation is critical for manufacturing quality control. Due to the scarcity of training defect samples, few-shot semantic segmentation (FSS) holds significant value in this field. However, existing studies mostly apply FSS to tackle defects on simple textures, without considering more diverse scenarios. This paper aims to address this gap by exploring FSS in broader industrial products with various defect types. To this end, we contribute a new real-world dataset and reorganize some existing datasets to build a more comprehensive few-shot defect segmentation (FDS) benchmark. On this benchmark, we thoroughly investigate metric learning-based FSS methods, including those based on meta-learning and those based on Vision Foundation Models (VFM). We observe that existing meta-learning-based methods are generally not well-suited for this task, while VFMs hold great potential. We further systematically study two types of VFMs, including upstream representation learning models and downstream SAM (Segment anything) series models. We propose a novel training-free FDS method, called FM-SAM, which is based on feature matching combined with FastSAM for refinement. It demonstrates convincing segmentation performance while maintaining high efficiency. In addition, we find that SAM2 can directly perform effective FDS end-to-end through its video tracking mode. The contributed dataset and code are available at: https://github.com/liutongkun/GFDS.