WSSS
Contract WSSS research · 53.31% COCO-Val mIoU · +1.5 pp over WeCLIP+
Built with PyTorch · CLIP · DINOv2 · Semantic segmentation · COCO-Val 2014
WHY
A client commissioned a study to improve WeCLIP+, then the state of the art, using only image-level labels. The pseudo-masks used for training accumulated wrong pixels at object boundaries and in the background, and those errors were amplified during self-training. The research question was whether only the unreliable pixels could be identified and repaired instead of rebuilding every mask.
Evaluation over the complete validation set
Final semantic-segmentation performance
Compared with WeCLIP+ at 51.8% mIoU
HOW
From weak labels to reliable pixel supervision
The model preserves strong pretrained representations and spends learning capacity on the uncertain parts of each generated mask.
- 01 Frozen representations
Extract complementary visual evidence with CLIP and DINOv2 without updating either backbone.
- 02 Reliability map
Measure where the two representations support the same pixel assignment and where they disagree.
- 03 Pixel repair
Replace only the unreliable regions instead of regenerating the entire pseudo-mask.
- 04 Segmentation training
Train the feature-fusion heads and segmentation decoder using the repaired masks as supervision.
RESULT
COCO-Val comparison
| Method | Supervision | mIoU |
|---|---|---|
| WeCLIP+ | Image-level labels | 51.80% |
| Refined pseudo-label method | Image-level labels | 53.31% |
Both figures use the full COCO-Val 2014 evaluation set.
Contribution
- Designed the pixel-reliability and selective-repair strategy around frozen CLIP and DINOv2 representations.
- Implemented the refinement and retraining pipeline, then evaluated the final model over the complete COCO-Val 2014 set.