Share
Export Citation
Vision-Language Guided Pseudo-Labeling for Multi-Label Waste Classification
Asrul B.E.W.
Proceedings 2026 International Conference on Current Research in Artificial Intelligence and Data Science Iccraids 2026
Abstract
Cost efficient multi-label waste recognition in canals is hindered by clutter, occlusion, and illumination variability, while exhaustive annotation remains costly. This paper proposes TGS a vision-language-guided pseudo-label curation framework that integrates a ConvNeXt-Tiny (Convolutional Neural Network Next Tiny) classifier with Grounding DINO (text-conditioned detection) and the Segment Anything Model (SAM) for spatial verification. In TGS, a ConvNeXt-Tiny teacher trained on a small labeled subset produces image-level confidences on unlabeled images; selected labels are converted into text prompts and verified by Grounding DINO to generate prompt-consistent regions, which are refined into pixel-level masks by SAM. The core novelty lies in SAM-based mask verification, which gates pseudo-label retention using pixel-level evidence rather than confidence-only or box-level cues. Pseudo-labels are retained only when region and mask evidence satisfy quality filters, then combined with human labels to train a student model within a Semi-Supervised Learning (SSL) loop. Experiments use a canal dataset from Makassar, Indonesia (855 images, 18 classes, 2,662 instances); after quality control, 769 images are retained, with a fixed 171image test set and labeled fractions of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$5 \%, 10 \%$</tex>, and 20 % sampled from a 598-image development pool. At <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1 0 \%}$</tex> labeled data <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(86 / 512 / 171)$</tex>, TGS achieves micro-averaged F1-score (micro-F1) 0.9393 with pseudo-label coverage 0.9199, improving reliability over naïve self-training (micro-F1 0.9376, coverage 0.9512) and outperforming FixMatch (micro-F1 0.9036). At 20 % labeled data, micro-F1 reaches 0.9581, approaching the fully supervised upper bound (0.9896). Future work will audit pseudo-label quality on a manually annotated subset, adopt class adaptive thresholds, and benchmark efficiency and robustness for real- time deployment.