AdaptMatch: Adaptive Consistency Regularization for Semi-supervised Learning with Top-k Pseudo-labeling and Contrastive Learning

Semi-supervised learning has been established as a very effective paradigm for utilizing unlabeled data in order to reduce dependency on large labeled datasets. Most of the semi-supervised learning (SSL) methods proposed recently rely on a predefined and extremely high threshold to select unlabeled data that contribute to the training, thus failing to consider different learning statuses of the model and feature learning from unlabeled data. To address this issue, we propose AdaptMatch, an adaptive learning approach to leverage unlabeled data using Top-k pseudo-labeling and contrastive learning according to the model’s learning status. The core of AdaptMatch is to adaptively adjust rules for different learning phases to allow informative unlabeled data and their pseudo-labels. If we cannot get high-confidence pseudo-labels from unlabeled data, contrastive learning can help the model learn more common features within the class. AdaptMatch outperforms or equals the state-of-the-art performance on a range of SSL benchmarks, exceptionally superior when the labeled data are extremely limited or imbalanced. For example, AdaptMatch reaches 91.56% and 97.44% accuracy with 4 labeled examples per class on CIFAR-10 and SVHN respectively, substantially improving over the previously best 88.70% and 96.66% accuracy achieved by FixMatch and ReMixMatch. Meanwhile, AdaptMatch also improves the accuracy of FixMatch in CIFAR10-LT with a performance gain of up to 2.3%.