justherozen / promix Goto Github PK
View Code? Open in Web Editor NEW[IJCAI 2023] ProMix: Combating Label Noise via Maximizing Clean Sample Utility
License: MIT License
[IJCAI 2023] ProMix: Combating Label Noise via Maximizing Clean Sample Utility
License: MIT License
Hello, thank you for your nice work.
I want to know how to generate or download CIFAR-10_human.pt
?
hi there,
老师请问,噪声的pt文件在哪里呢?
Because ProMix trains two equivalent networks, there are a few minor issues to consider when it comes to inference:
a) Considering the balance between inference speed and performance, should I take the mean of the two networks or just one of them?
b) Is there a robustness issue with the network trained using weak data augmentation?
Additionally, regarding the training process, there are a few minor concerns:
c) What impact would using two networks with different architectures have on the algorithm?
d) How should difficult samples (correctly labeled but with larger loss and mispredictions) in the training set be further handled?
I appreciate your help.
cifar10:clean_label | Epoch [ 10/ 40] Iter[ 1/782] Net1 loss: 0.95 Net2 loss: 1.22
Traceback (most recent call last):
File "E:\pycharmproject\weak_supervised\ProMix-main\ProMix-main\Train_cifarn.py", line 463, in
train(epoch, dualnet.net1, dualnet.net2, optimizer1, total_trainloader, unlabeled_trainloader)
File "E:\pycharmproject\weak_supervised\ProMix-main\ProMix-main\Train_cifarn.py", line 181, in train
loss_fmix = fmix.loss(logits_fmix, (pseudo_label_c.detach()).long())
File "E:\pycharmproject\weak_supervised\ProMix-main\ProMix-main\fmix.py", line 94, in loss
return fmix_loss(y_pred, y, self.index, self.lam, train, self.reformulate)
File "E:\pycharmproject\weak_supervised\ProMix-main\ProMix-main\fmix.py", line 46, in fmix_loss
y1 = y1.max(1)[1]
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity
train_promix.py at line 437:
#class-wise small-loss selection (CSS for base selection set)
losses = (losses - losses.min()) / (losses.max() - losses.min())
all_loss.append(losses)
input_loss = losses.reshape(-1, 1)
prob = np.zeros(targets_list.shape[0])
idx_chosen_sm = []
min_len = 1e10
for j in range(num_class):
indices = np.where(targets_list.cpu().numpy()==j)[0]
if len(indices) == 0:
continue
bs_j = targets_list.shape[0] * (1. / num_class)
pseudo_loss_vec_j = losses[indices]
sorted_idx_j = pseudo_loss_vec_j.sort()[1].cpu().numpy()
partition_j = max(min(int(math.ceil(bs_j*rho)), len(indices)), 1)
idx_chosen_sm.append(indices[sorted_idx_j[:partition_j]])
min_len = min(min_len, partition_j)
idx_chosen_sm = np.concatenate(idx_chosen_sm)
prob[idx_chosen_sm] = 1
When the data set is extremely imbalanced, it seems unreasonable that this average selection exceeds the total number of minority classes.
For example, if the number of categories is 100, 10, the average selection is 50*0.5=25,25>10.
How is this part set up in the category imbalance experiment?
Which file needs to be run to reproduce the experiment?
我在寻找一种可以解决以下问题的方法:
因为人失误造成的错标,或者说模棱两可的数据标成任意一个类都可以,这样产生的数据集本身存在问题。
可是我的分类任务可能差异性并不是很明显,属于细颗粒度的分类,不像狗和茶杯这样的类别,而是OK和NG这样的判定,可能稍微有一点裂缝没啥问题,裂缝太大才有问题,稍微有一点缺失没问题,缺失很大才有问题,您的这个方法可以应用在我这个任务上吗?像我这种二分类是不是使用回归的方法效果更好哇,希望能得到您的回复。
When warmup training, the model performance is very poor and the training does not seem to work.
my log:
------------ warmup satrt ----------------
Epoch:0 Accuracy:8.86
Epoch:1 Accuracy:6.71
Epoch:2 Accuracy:6.63
Epoch:3 Accuracy:7.84
Epoch:4 Accuracy:10.38
Epoch:5 Accuracy:10.47
Epoch:6 Accuracy:12.23
Epoch:7 Accuracy:13.41
Epoch:8 Accuracy:12.16
Epoch:9 Accuracy:13.13
------------ warmup end -----------------
Epoch:10 Accuracy:73.77
Epoch:11 Accuracy:75.59
Epoch:12 Accuracy:76.33
Epoch:13 Accuracy:76.40
cifar10 has 10 classes, avg acc = 10
Model performance also tends to be average when training with warmup on my dataset.
In addition, another bug is reported. When idx_unchosen is empty, the loss calculation is nan. Perhaps it should be judged that when idx_unchosen is empty, the pseudo loss is 0.
on train_promin.py
# inputs_x: strong augmentation
# inputs_x2: weak augmentation
Code comments and actual code execution are exactly opposite. What should be the correct one? Is the comment written wrong?
I saw in the paper that you also achieved good results on clothing1m. I tried to reproduce it myself. The final good result was 94.01%. There may be a problem with my code, so I would like to ask you about the details of your experiments on clothing1m.
Hi, I'm Youngjae Kim.
I'm doing an experiment with custom data.
After a certain point (e.g. Epoch 37 Iter 5), the loss is still coming out as nan.
Has this ever happened to you?
If so, how did you solve it?
CIFAR10 seems to run but when I take CIFAR100 it gives this error?
Any idea.Is there any problem with the file 100_human.pt](https://github.com/UCSC-REAL/cifar-10-100n/blob/main/data/CIFAR-100_human.pt)
I see models are different
Hi there , just tested the model on my dataset and the performance during warm up is good, but it decreases a lot during training. Acutally the test metrics doesn;t change. Seems like the trained model is not used at all. The loss is performing good. Could you please share any ideas? Much appreciated.
Warmup:
| Building net
save noisy labels to ./data/Wafer/noise_file/clean_label_058597.json ...
Warmup Net1
Wafer:clean_label | Epoch [ 0/600] Iter[ 1/104] CE-loss: 7.5067 Penalty-loss: -7.1986 All-loss: 7.5067
Wafer:clean_label | Epoch [ 0/600] Iter[101/104] CE-loss: 1.7427 Penalty-loss: -1.7540 All-loss: 1.7427
| Test Epoch #0 Acc Net1: 0.18%, Acc Net2: 2.74% Acc Mean: 4.44%
Warmup Net1
Wafer:clean_label | Epoch [ 1/600] Iter[ 1/104] CE-loss: 1.5454 Penalty-loss: -1.7752 All-loss: 1.5454
Wafer:clean_label | Epoch [ 1/600] Iter[101/104] CE-loss: 1.0084 Penalty-loss: -1.2549 All-loss: 1.0084
| Test Epoch #1 Acc Net1: 3.45%, Acc Net2: 1.19% Acc Mean: 70.52%
Warmup Net1
Wafer:clean_label | Epoch [ 2/600] Iter[ 1/104] CE-loss: 0.7619 Penalty-loss: -0.9628 All-loss: 0.7619
Wafer:clean_label | Epoch [ 2/600] Iter[101/104] CE-loss: 0.9429 Penalty-loss: -1.0692 All-loss: 0.9429
| Test Epoch #2 Acc Net1: 2.54%, Acc Net2: 1.53% Acc Mean: 84.39%
after:
Wafer:clean_label | Epoch [ 3/600] Iter[ 82/104] Net1 loss: 0.74 Net2 loss: 0.85
Wafer:clean_label | Epoch [ 3/600] Iter[ 83/104] Net1 loss: 0.88 Net2 loss: 0.84
Wafer:clean_label | Epoch [ 3/600] Iter[ 84/104] Net1 loss: 0.73 Net2 loss: 0.94
Wafer:clean_label | Epoch [ 3/600] Iter[ 85/104] Net1 loss: 0.84 Net2 loss: 0.87
Wafer:clean_label | Epoch [ 3/600] Iter[ 86/104] Net1 loss: 0.74 Net2 loss: 0.61
Wafer:clean_label | Epoch [ 3/600] Iter[ 87/104] Net1 loss: 0.69 Net2 loss: 0.70
Wafer:clean_label | Epoch [ 3/600] Iter[ 88/104] Net1 loss: 0.76 Net2 loss: 0.87
Wafer:clean_label | Epoch [ 3/600] Iter[ 89/104] Net1 loss: 0.73 Net2 loss: 0.79
Wafer:clean_label | Epoch [ 3/600] Iter[ 90/104] Net1 loss: 0.61 Net2 loss: 0.81
Wafer:clean_label | Epoch [ 3/600] Iter[ 91/104] Net1 loss: 0.72 Net2 loss: 0.87
Wafer:clean_label | Epoch [ 3/600] Iter[ 92/104] Net1 loss: 0.92 Net2 loss: 0.92
Wafer:clean_label | Epoch [ 3/600] Iter[ 93/104] Net1 loss: 0.79 Net2 loss: 0.78
Wafer:clean_label | Epoch [ 3/600] Iter[ 94/104] Net1 loss: 0.80 Net2 loss: 0.83
Wafer:clean_label | Epoch [ 3/600] Iter[ 95/104] Net1 loss: 0.65 Net2 loss: 0.76
Wafer:clean_label | Epoch [ 3/600] Iter[ 96/104] Net1 loss: 0.64 Net2 loss: 0.57
Wafer:clean_label | Epoch [ 3/600] Iter[ 97/104] Net1 loss: 0.71 Net2 loss: 0.69
Wafer:clean_label | Epoch [ 3/600] Iter[ 98/104] Net1 loss: 0.83 Net2 loss: 0.65
Wafer:clean_label | Epoch [ 3/600] Iter[ 99/104] Net1 loss: 0.57 Net2 loss: 0.73
Wafer:clean_label | Epoch [ 3/600] Iter[100/104] Net1 loss: 0.72 Net2 loss: 0.72
Wafer:clean_label | Epoch [ 3/600] Iter[101/104] Net1 loss: 0.84 Net2 loss: 0.90
Wafer:clean_label | Epoch [ 3/600] Iter[102/104] Net1 loss: 0.70 Net2 loss: 0.50
Wafer:clean_label | Epoch [ 3/600] Iter[103/104] Net1 loss: 0.75 Net2 loss: 0.79
| Test Epoch #3 Acc Net1: 2.63%, Acc Net2: 2.63% Acc Mean: 2.63%
The cifar10 IDN 0.2 and 0.4 noise rate, I can only get about 93.6%-94% acc useing the code, and I can't find the reason. Could you tell me which hyper-parameters you use to train?
Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.