google-research / noisystudent

Code for Noisy Student Training. https://arxiv.org/abs/1911.04252

Google Research

This repository contains code released by Google Research.

All datasets in this repository are released under the CC BY 4.0 International license, which can be found here: https://creativecommons.org/licenses/by/4.0/legalcode. All source files in this repository are released under the Apache 2.0 license, the text of which can be found in the LICENSE file.

---

Because the repo is large, we recommend you download only the subdirectory of interest:

SUBDIR=foo
svn export https://github.com/google-research/google-research/trunk/$SUBDIR

If you'd like to submit a pull request, you'll need to clone the repository; we recommend making a shallow clone (without history).

git clone [email protected]:google-research/google-research.git --depth=1

---

Disclaimer: This is not an official Google product.

Updated in 2023.

noisystudent's Issues

Why this code takes only a small part of GPU mem and volatile only 0%

I'm trying to train bash local_scripts/svhn/run.sh on my GPU (titan rtx).
I don't know why this only takes 161MB of the GPU and GPU-Util (volatile) 0%.
The output process: examples/sec: 10.5986
Anyone know why is this ?

the result got 0.09 about top_1_accuracy and got 0.50 about top_5_accuracy

hi ,
i run the "bash local_scripts/svhn/run.sh".but the result is poor, however, the teacher model trained on labeled data with accuracy 97.9. so ,why happened to this ?

Which tensorflow version are you using?

Hi guys, thanks for the very interesting work!

Could please provide the list of required packages? I am just trying to reproduce the experiments. Thanks!

How to train on BDD100K data?

Hi,

Please guide me how to train this model on custom data set or other standard data set like BDD100K?

Thanks,

Test accuracy?

测试预训练模型：
I1020 09:23:09.595353 140204815505152 main.py:805] test, results: {'loss': 2.5957024, 'top_1_accuracy': 0.0988783, 'top_5_accuracy': 0.5069914, 'global_step': 0}
与提供97.9%对比不上？

HOW

Hi！How to use your trained model to test my own dataset？Thank you.

How to train EfficientNet based ArcFace model with Noisy Student method

Hello
How are you?
Thanks for contributing this project.
I have a question.
Could u please explain how to train EfficientNet based ArcFace model with noisy student method?
Thanks

Confusion about the loss function

hello, when i look at this code after read the paper,I am quite confused about how the loss function is calculated in the code.The final loss in the paper is the sum of the cross-entropy loss of the distribution calculation of the two kinds of data (with or without labels), but the calculation method in the code is not the same. like:
real_lab_bsz = tf.to_float(lab_bsz) * FLAGS.label_data_sample_prob
real_unl_bsz = batch_size * FLAGS.label_data_sample_prob * FLAGS.unlabel_ratio
data_loss = lab_loss * real_lab_bsz + unl_loss * real_unl_bsz
data_loss = data_loss / real_lab_bsz`
The loss in front of this part of the code has been averaged, but it has multiplied by the number of samples first, and then divided by the number of labeled samples. What is the significance of this calculation method? It doesn't feel the same as described in the paper.

Can these hyperparameters: layer number, layer dimensions, etc., trainable by this noiseStudent?

From what I read, the noisystudent is to get the layer weights from data, but could these architecture parameters trainable using this semi-supervision methods?

techer model link not working

Hi, I am trying to run your script on colab, but the error message shows that the URL for pulling teacher_ckpt is not working, is there a new link to the teacher model?

What is the structure of the training data for ImageNet?

Thank you for your great work! I am trying to re-do the experiment on ImageNet data as described in the Readme file.

However, I could not find anywhere in the instruction that shows how the folders label_data_dir and unlabel_data_dir are structured. Could you please clarify? Thank you!

tabular data/ noisy instances

Hi,
thanks for sharing your implementation. I have two questions about it:

1. Does it also work on tabular data?
2. Is it possible to identify the noisy instances (return the noisy IDs or the clean set)?

Thanks!