Due to poor bias calibration, current semi-supervised learning (SSL) is over-confident in false predictions. Modeling data noise helps alleviate confirmation bias. Specifically, we consider a posteriori knowledge that the data embedding follows a Gaussian distribution. We let the model predict and learn the mean (classification features) and variance (noise features) for each sample. In this way, the network can estimate and isolate noisy information in the latent space. Additionally, we propose Noise Estimation Curriculum Learning (NECL) to fit differences in noise level between classes. NECL can encourage the model to learn from hard classes actively. Extensive experiments demonstrate the effectiveness of our method, which is also orthogonal to FixMatch-based frameworks. InfoMatch with our components (NE-Info) showed state-of-the-art (SOAT) performance on several benchmarks such as CIFAR-10/100, SVHN, STL-10 and ImageNet.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Weaken Noisy Feature: Boosting Semi-supervised Learning by Noise Estimation

  • Junjiang Liu,
  • Dandan Sun,
  • Hailun Xia,
  • Jiangtao Bai,
  • Xinyue Fan

摘要

Due to poor bias calibration, current semi-supervised learning (SSL) is over-confident in false predictions. Modeling data noise helps alleviate confirmation bias. Specifically, we consider a posteriori knowledge that the data embedding follows a Gaussian distribution. We let the model predict and learn the mean (classification features) and variance (noise features) for each sample. In this way, the network can estimate and isolate noisy information in the latent space. Additionally, we propose Noise Estimation Curriculum Learning (NECL) to fit differences in noise level between classes. NECL can encourage the model to learn from hard classes actively. Extensive experiments demonstrate the effectiveness of our method, which is also orthogonal to FixMatch-based frameworks. InfoMatch with our components (NE-Info) showed state-of-the-art (SOAT) performance on several benchmarks such as CIFAR-10/100, SVHN, STL-10 and ImageNet.