Abstract:
Existing weakly or semi-supervised semantic segmentation methods often utilize image or box-level supervision to generate pseudo-labels for weakly labeled images. However, due to the lack of strong supervision, the generated pseudo-labels are often noisy near the object boundaries, which is critical for learning strong representations within semantic regions. To address this problem, we present a novel framework that generates pseudo-labels for training images, which are then used to train a multi-task segmentation model. To generate pseudo-labels, we combine information from: (i) a class agnostic `objectness’ network that learns to recognize object-like regions, and (ii) either image-level or bounding box annotations. We show the efficacy of our approach by demonstrating how the objectness network can naturally be leveraged to generate object-like regions for unseen categories. We then propose an end-to-end multi-task learning strategy, that jointly learns to segment semantics and objectness using the generated pseudo-labels. Extensive experiments demonstrate the high quality of our generated pseudo-labels and effectiveness of the proposed framework in a variety of domains. Our proposed approach achieves better or competitive performance compared to existing weakly-supervised and semi-supervised methods. The code and trained models will be released upon acceptance.
