hfslyc / advsemiseg Goto Github PK
View Code? Open in Web Editor NEWAdversarial Learning for Semi-supervised Semantic Segmentation, BMVC 2018
Adversarial Learning for Semi-supervised Semantic Segmentation, BMVC 2018
Hello ,
How can i train my own dataset using custom classes?
hi, thanks for your work.
when i run train.py, the error occurs : file"train.py",line 375,in main
loss_seg_value+=loss_seg.data.cpu( ).numpy()[0]/args.ier_size
indexError: too many indices fro array
i don't know how to solve it ,please help me
@hfslyc
设置梯度回传为False,这个位置不对。按照你的代码推理,对抗过程应该不能设置梯度为False。
for param in model_D.parameters():
param.requires_grad = True
pred = pred.detach()
您可以给出解释么?我在语义分割对抗方面是小白。
hello, when I try ti run train.py, an error accurs: ImportError: No module named packaging
i use python2.7 pytorch=0.4.0 cv2=3.4.2
can you please tell me how to solve it
thanks very much
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
How can i run it on cpu.Hlep me,please.Thank you!
Hello, I'm very sorry. Now that I understand that the gradient is set to false in your code, I've always understood that adv is in D, and it's actually in G. I know that the hyperparameters should be modified based on the data set. Since I misunderstood the "confrontation", I misunderstood your code and I'm sorry. I consulted with others before I discovered my mistake. Thank you for your reply.
File "D:\workspace\AdvSemiSeg\dataset\voc_dataset.py", line 56, in getitem
img_h, img_w = label.shape
AttributeError: 'NoneType' object has no attribute 'shape'
Can you help me? I tried some ways,but it still has errors.Thank you!
Hello!
During training loss_D value decrease to ~0.2 and segmentation network cant learn nothing. Does it mean that discriminator dominate under generator? what range of loss_d value is normal(for classical gan is 0.69) )? Maybe it's possible to attach some loss graphics?
Hello, in your implementation, you first train G network, then train on D network. The code can be summarized as
#=========Train on G=========
for param in model_D.parameters():
param.requires_grad = False
loss_seg = loss_calc(pred, labels, args.gpu)
loss_adv_pred = bce_loss(D_out, make_D_label(gt_label, ignore_mask))
loss = loss_seg + args.lambda_adv_pred * loss_adv_pred
loss.backward()
#=======Train on D===========
for param in model_D.parameters():
param.requires_grad = True
pred = pred.detach()
#-----Fake-------
loss_D = bce_loss(D_out, make_D_label(pred_label, ignore_mask))
loss_D.backward()
#-----Real--------
loss_D = bce_loss(D_out, make_D_label(gt_label, ignore_mask_gt))
loss_D.backward()
optimizer.step()
optimizer_D.step()
However, follow the DCGAN example, I make more simple way that does not requires setting param.requires_grad = False/True
. This is my modified
#=======Train on D===========
#-----Real--------
loss_D = bce_loss(D_out, make_D_label(gt_label, ignore_mask_gt))
loss_D.backward()
#-----Fake-------
pred= model(image)#add more here
pred = pred.detach()
loss_D = bce_loss(D_out, make_D_label(pred_label, ignore_mask))
loss_D.backward()
optimizer_D.step()
#=========Train on G=========
loss_seg = loss_calc(pred, labels, args.gpu)
loss_adv_pred = bce_loss(D_out, make_D_label(gt_label, ignore_mask))
loss = loss_seg + args.lambda_adv_pred * loss_adv_pred
loss.backward()
optimizer.step()
In this modification, I trained the D network first with real label and detach
for the fake label. We did not use param.requires_grad = False/True
because detach
did it. The code is enabled to run. However, I did not guarantee it equivalent with your purpose or not "During the training G network, we fix the D network`. Could you give me some comment?
Reference :https://discuss.pytorch.org/t/when-training-gan-why-do-we-not-need-to-zero-grad-discriminator/20439
In the update step of the discriminator (line 208), the generator does not get the data, so the backward step does not calculate any gradients for it.
In line 217 the input to the discriminator is detached as you already observed. Thus the backward call of errD_fake also does not calculate the gradients for the generator.
Before updating the generator (line 225 etc.) the gradients are zeroed, so it looks alright
Hello, first of all, thank you for sharing your work. I have a question in the process of looking at the code.ignore_mask = (y.numpy() == 255) Can you explain this to me? If my label y has a value of 0-9, can it be written like this? I look forward to your reply. Thank you very much.
hello, can you please tell me what the [ def make_D_label(label, ignore_mask): ]mean in train.py line 175-179.
thanks
It is not a bug. I just want to ask this question to make more clearly understand.
Your discriminator network produces a spatial map by using fully convolution layer.
Hi
Thank you for release the code and answer my question.
how can I train a new advFull model ?
just set PARTIAL_DATA=1? in train.py
I have a question with the choice of mask_T. How did you choose its value? I think 0.2 is too small. Maybe larger value may be better?
Hey ,
When I try to validate the results , I am getting this error
output = model(Variable(image, volatile=True).cuda(gpu0))
Traceback (most recent call last):
File "evaluate_voc.py", line 235, in <module>
main()
File "evaluate_voc.py", line 214, in main
output = model(Variable(image, volatile=True).cuda(gpu0))
File "/home/ubuntu/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/summer_interns/jose/AdvSemiSeg/model/deeplab.py", line 167, in forward
x = self.conv1(x)
File "/home/ubuntu/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDNN_STATUS_MAPPING_ERROR
Hello,thanks for your work. In your article ,you said your remove the last classification layer and use the Atrous Spatial Pyramid Pooling method in the last layer. But in your deeplab.py, it use the classification_module and no mention to the ASPP method. I can not understand why?Can you replay me?...please
Hi,
Thanks for sharing your work!
In your code:
if (args.lambda_semi > 0 or args.lambda_semi_adv > 0) and i_iter >= args.semi_start_adv:
as far as i understand, you want to check whether to train the generator with unlabeled data(so that lambda_semi or semi_adv should be larger than 0), and also you check if semi_start_adv is smaller than iteration number. If you do that, inside if case would be run just once. I think it should be i_iter <= args.semi_start_adv, so that once starting number of semi-adversarial training more than number of iterations that have been run, generator weights will be updated with adversarial loss and self thought learning?
Am I missing something?
I appreciate your answer, thanks! Ecem
I got RuntimeError: CUDNN_STATUS_EXECUTION_FAILED when executing evaluation_voc.py
My conda list is below
python 2.7.17 h9bab390_0
python-dateutil 2.8.1 py_0
pytorch 0.4.0 py27hdf912b8_0
cudatoolkit 9.0 h13b8566_0
cudnn 7.6.4 cuda9.0_0
The weird thing is that train.py, which uses the same modules, works well with the same environment.
P.S. If the version of pytorch is upper than 0.4, the following adjustment is required.
with torch.no_grad():
for index, batch in enumerate(testloader):
if index % 100 == 0:
print('%d processd'%(index))
image, label, size, name = batch
size = size[0].numpy()
output = model(image.cuda(args.gpu))
output = interp(output).cpu().data[0].numpy()
Is't possible to start training in multi gpu mode? seems that it's not possible.
parser.add_argument("--gpu", type=int, default=0,
help="choose gpu device.")
I don't understand why you use the VOCGTDataSet instead of VOCDataSet , they choose the same data from voc train dataset :train_ids[:partial_size],,,I think the difference between these two class is whether to pad the image and label,
first:
ignore_mask = (labels.numpy() == 255)
ignore_mask_gt = (labels_gt.numpy() == 255)
then:
D_out = interp(model_D(F.softmax(pred)))
loss_D = bce_loss(D_out, make_D_label(pred_label, ignore_mask))
D_gt_v = Variable(one_hot(labels_gt)).cuda(args.gpu)
D_out = interp(model_D(D_gt_v))
loss_D = bce_loss(D_out, make_D_label(gt_label, ignore_mask_gt))
I think these two loss_D is already have different D_out,is it necessary to distinguish VOCGTDataSet and VOCDataSet ?
Hello, I want to ask one question related to the loss_adv_p
and loss_D
. As the log, the loss_adv_p
tends to increase while the loss_D
tends to decrease. We expect that loss_D
will decrease, it is correct; however, how about the loss_adv_p
? I found one interesting thing that is args.lambda_adv_pred
, you can set it to 0.5
, then the loss_adv_p
will decrease but the loss loss_D
will increase. They looks opposite side Thanks
iter = 0/ 20000, loss_seg = 3.045, loss_adv_p = 0.695, loss_D = 0.693, loss_semi = 0.000
and at 180 iters
iter = 180/ 20000, loss_seg = 0.490, loss_adv_p = 1.358, loss_D = 0.572, loss_semi = 0.000
Hi, I am wondering why you use trainloader_iter and trainloader_gt_iter at the same time during your training process, since they are both refer to the labeled dataset. During the D training, only labels_gt is used from trainloader_gt_iter, what is the difference if using labels from trainloader_gt_iter?
I would like to ask what is the use of utils.metric.py?
Excuse me, There are one more questions...hope it not bothing you too much...
"The output of D is a probability map with values ranges from 0-1. When the input is unlabeled data, we use the D output map as the indicator for semi-supervised learning."
D_out = interp(model_D(F.softmax(pred)))
D_out_sigmoid = F.sigmoid(D_out).data.cpu().numpy().squeeze(axis=1)
##following is used to draw the confindence map in your papar Figure3
for i in range(args.batch_size):
plt.imshow(D_out_sigmoid[i], cmap='jet')
plt.show()
Can I interpret the D_out_sigmoid is the confidence map you mentioned?
I use VOC_20000.pth and VOC_20000_D.pth(when use these two model the evaluatation mIOU is 0.709) as the parameters for RESTORE_FROM and RESTORE_FROM_D,and tried to get the figure of confidence map,but it looks not matching so well,it looks weird ,can you explain how you get the confidence map in your paper?
Hello, this is not the bug in your code. I just want to ask a question about my code
I used your code in my dataset. I found that both the loss_adv
and loss_D
tend to decrease. While the correct trend must be loss_adv
increases and loss_D
decrease. What is my problem? Could you give me some suggestions? Thanks
想请问一下在训练得到confidence map过程中判别器的输出map大概是怎样的呢,我现在一直找不到分割网络和判别网路的训练平衡点,我做的肝脏病灶分割,分割网络预先训练使其达到一定的分割效果才加的判别器,判别器loss会降到很低,加上判别器分割精度会下降,请问您对我的训练有什么建议吗
Hi! First, I'd like to thank you for the very helpful repo. During training with the same discriminator as yours, I have met some problems which I can't figure out. 1. The discriminator loss of pred and GT are almost unchanged for most of the time. So I am wondering if this phenomenon is normal. Btw, I haven't added semi data yet.
Thanks for your great work. Did you implement the code to reproduce the experiments in Cityscape dataset in the paper? Thanks!
Thanks for sharing.i have got a issure about command parameters.
write thispython evaluate_voc.py --help
in terminal,and thers is a optional --num-class
the original code is:NUM_CLASSES = 21
,i wonder what will happen if i code NUM_CLASSES = 3
.
what 's the order of the object .is it random or has a default order?
In this paper, the discriminator network takes a probability map of size H × W × C and outputs a confidence map of size H × W × 1. I don't understand how to train this discriminator network, what are the labels during the training phase ? As my understanding, when the probability map comes from the ground truth, the label is a map of size H × W × 1 and the values in the map are all 1, in contrast, when the probability map comes from the segmentation network, the label is a map of size H × W × 1 and the values in the map are all 0. Am I right? Hope to get your answer. Thanks so much
Thanks for sharing your work. I used evaluate code and I got the error.Here is the detail:
Traceback (most recent call last):
File "evaluate_voc.py", line 236, in
main()
File "evaluate_voc.py", line 210, in main
for index, batch in enumerate(testloader):
File "C:\Users\landlord\Anaconda3\envs\aa\lib\site-packages\torch\utils\data\dataloader.py", line 264, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "C:\Users\landlord\Anaconda3\envs\aa\lib\site-packages\torch\utils\data\dataloader.py", line 264, in
batch = self.collate_fn([self.dataset[i] for i in indices])
File "E:\AdvSemiSeg-master\dataset\voc_dataset.py", line 58, in getitem
img_h, img_w = label.shape
> AttributeError: 'NoneType' object has no attribute 'shape'
I am using python 3.6 from conda install, win10. How can I fix it? Thanks
Hi, when I use your code by your README, I find that the model training is fintuning on your well-trained model. When I choose not to fintuning , and want to retrain the data VOC2012. Some wrongs with the loss. The losses don't converge. I want to konw if anything wrong with your code? emmm, I with you to test your code. Thanks.
Thanks for sharing your work. I used evaluate code and I got the error
import copy_reg
ModuleNotFoundError: No module named 'copy_reg'
I am using python 3.6 from conda install, Ubuntu 16.04. How can I fix it? Thanks
loss_semi_adv_value += loss_semi_adv.data.cpu().numpy()[0]/args.lambda_semi_adv
IndexError: too many indices for array
I want to run the code for "DeepGlobe Land Cover Classification Challenge dataset". Please explain me, what are the changes I need to make in the code.
Thanking you
You have three losses: loss_seg, loss_Adv and loss_semi. Each loss used batch size of 10. So, what is batch size of combined loss=loss_seg + alpha *loss_Adv +beta*loss_semi
. Is it still 10 or 30? Thanks
Hello, I am planning to use your data to run your network but it fails. My data is in two categories. One is for the background and the other is set to 0 and 1, respectively. For the single channel label map, use your The network has been trained, but the test result output is all 0. May I ask if I need to adjust some parameters, thank you
Hi there,
First, impressive work!
I'm reading your code but find one uncertain part: what's SEMI_START and SEMI_START_ADV represented for? I know it's used during training semi to determine the #iterations starting training. However in the original code, SEMI_START=5000 and SEMI_START_ADV=0, which means that semi is trained at the very FIRST iteration. Is that a bug since in the paper you mentioned "we start semi-supervised learning after training for 5000 iterations with labeled data"?
Any comments are appreciated!
Hi,
Thanks again for sharing your code, your work is very interesting.
I see in your paper you mention that the discriminator is only trained with labeled data. However, you assign D_remain argument to True. And in the instructions you give in order to train the network for reproducing your results, you do not set it to False. Am i wrong?
Thanks,Ecem
when I train the model on VOC2012, it stops at the 160/20000 .
Traceback (most recent call last):
File "train.py", line 474, in
main()
File "train.py", line 335, in main
_, batch = trainloader_remain_iter.next()
File "/home/zhengkai/anaconda3/envs/tensorflow/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 572, in next
raise StopIteration
StopIteration
Hi,
Thanks for sharing your code, i find your work very interesting!
I would like to use your work for my research, I have been checking your code, and as far as i understand, you have a function make_D_labels which return a matrix with all values 1 in case they come from GT or 0 when they come from generator.
However, you have an ignore_mask which you use it for assigning some part of D_label matrix to 255 . I do not understand this. Why would you assign 255 to any item of the matrix? Could you explain what i am missing?
Thanks for your help, I appreciate it!
Regards,Ecem
In "read me", you commented that the mean IOU should be around 68.8% for the pre-trained model for 0.125 set (adv+ semi).
However, in the paper the result is 69.5% other than 68.8%.
Why did you comment that it is around 68.8%? (Actually, the pre-trained weight from you got 69.4% in my computer, not around 68.8%, which is similar with the result of the paper.)
(In addition, when I retrained your code with the batch size of 8 for RTX 2080 Ti.
The result was 68.5%, which is around 68.8%, but I think this is because of the batch size.)
It seems loss_semi_adv&&loss_adv_pred alwayes contribute to the network during training,
I understandt that :
loss_adv_pred = bce_loss(D_out, make_D_label(gt_label, ignore_mask))
is using the gt_label,cause the pred is come from labeled data,but
loss_semi_adv = args.lambda_semi_adv * bce_loss(D_out, make_D_label(gt_label, ignore_mask_remain))
why this loss also using the gt_label,the data is come from the trainloader_remain_iter...?
When I run the evaluation code, I get a mean IoU of 47% as against 68.8% as mentioned. This was done using pretrained model using 1/8 training data. Is there something that I have done wrong or some changes that have to be made to the code before running evaluation? The dataset was VOC2012.
In line 371 of script train.py:
loss = loss_seg + args.lambda_adv_pred * loss_adv_pred
It seems to me that the loss_adv_pred
part doesn't do anything for the training, since you have set require_gradient
for the discriminator to False in line 309-311:
# don't accumulate grads in D
for param in model_D.parameters():
param.requires_grad = False
Am I thinking it wrong? If so, could you please tell me about it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.