Multi-class Token-Guided End-to-End Weakly Supervised Image Semantic Segmentation Method

Weakly supervised image semantic segmentation has become the most popular method in recent years because of its low cost and has been widely used in medical image segmentation, automatic driving, remote sensing image analysis and other fields. However, the current weakly supervised semantic segmentation based on transformer has some problems, such as focusing on the whole, ignoring local details and confusing different categories. To solve these problems, we come up with a token-guided single stage weakly supervised image semantic segmentation algorithm. First of all, in order to solve the problem of insufficient attention to details, we proposed an optimization clipping method, which realized the selection of uncertain regions as much as possible and the fine marking of uncertain regions. Then, the single-class token to multiple class tokens method is purposed to obtain multiple class tokens for fine guidance. In particular, we designed a multiple class tokens guide method to complete the function of classifying uncertain regions and correctly activating them. The quantitative and qualitative results of the public dataset PASCAL VOC 2012 validate the effectiveness of the method.