twke18 / adaptive_affinity_fields Goto Github PK

Thanks for sharing the code. For the multiple gpu version, in the train_mpgu.py file
"from vocseg.models.pspnet_mgpu import pspnet_resnet101 as model", it seems the vocseg files are missing. Could you please update the files? Many thanks.

Excuse me, if "aff_loss_edge" and "aff_loss_nedge" do not have a coefficient of "dec" that decreases with "step", can "aff_loss" converge? When I run "train_affinity.py" with the parameters mentioned in "train_pspnet_affinity.sh", "aff_loss_edge" will not fall until 2.88.

We downloaded the data set and model parameters in your steps, but get the value of loss is Nan at run time. here is print ------ loss = nan, lr = 0.001000: 0%| | 1/200 [00:49<2:44:03, 49.46s/it]，
I would appreciate it if you could reply to me.

Hi，
Could you provide your training model on voc 2012? I didn't find him.
Thank you very much!

I was trying to understand how you implement AAF. However, I found it difficulty to understand how it exactly works. Specifically, I am not sure what do you mean in the comments of ignores_from_label by stating that 'Retrieve eight corner pixels from the center, where the center is ignored. Note that it should be bi-directional'. Why it should be bi-directional?

Thanks in advance for your help!

As the comments in the function of "ignores_from_label" in layers.py say:

"""Retrieves ignorable pixels from the ground-truth labels. This function returns a binary map in which 1 denotes ignored pixels and 0 means not ignored ones. For those ignored pixels, they are not only the pixels with label value >= num_classes, but also the corresponding neighboring pixels, which are on the the eight cornerls from a (2size+1)x(2size+1) patch.

In my option, it means that it will filter some invalid labels in the input(e.g the 255 in Pascal VOC).
But it seems like that the code is not correct. I made some simple attempts:

N = s * s - 1
tensor = tf.constant(np.random.randint(0, 2, 25, dtype=np.int32), dtype=tf.int32)
tensor = tf.reshape(tensor, (1, 5, 5))
ignore = ignores_from_label(tensor, num_classes=5, size=1)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(np.reshape(sess.run(tensor), (5, 5)))
    print('------------------')
    print(tensor.get_shape().as_list())
    out = np.reshape(sess.run(ignore), (5, 5, N))
    for n in range(N):
        print(n)
        print(out[:, :, n].astype(np.int32))
            print(n)
            print(out[:, :, n].astype(np.int32))

the output is:

[[1 1 0 1 0]
 [0 1 1 0 0]
 [0 0 1 0 0]
 [1 1 1 1 0]
 [1 0 1 1 0]]

[1, 5, 5]
0
[[1 1 1 1 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 1 1 1 1]]
1
[[1 1 1 1 1]
 [0 0 0 0 0]
 [0 0 0 0 0]
 [0 0 0 0 0]
 [1 1 1 1 1]]
2
[[1 1 1 1 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 1 1 1 1]]
3
[[1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]]
4
[[1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]]
5
[[1 1 1 1 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 1 1 1 1]]
6
[[1 1 1 1 1]
 [0 0 0 0 0]
 [0 0 0 0 0]
 [0 0 0 0 0]
 [1 1 1 1 1]]
7
[[1 1 1 1 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 0 0 0 1]
 [1 1 1 1 1]]

The result is unexpected. Does it means to ignore the boundary pixels of the image?

The inference code could not run!can you give some default input parameter,so that I could just set the input output data directory and list!Thank you.

Hi，should I change the folder from VOCdevkit/VOC2012/SegmentationClass to VOCdevkit/VOC2012/segcls? Because there is no segcls folder in the dataset VOC2012.

Hi, in the original implementation there are multiple labels (background, label 1, labels 2, etc..). In my application I have only one class besides background. Therefore I use sigmoid and have logits in the shape (B, H, W, 1). Should I expand my labels and predictions to (B, H, W, 2) where labels/pred[..., 1] = 1 - pred[..., 0]?

Or can I just compute for a single channel since background and foreground are complementary for the binary case?

Thank you,
Kind regards

Hi, I've adapted the code here for Pytorch and it seems to have worked ok. However I'm experience a huge overhead in terms of time. Without AAF a single epoch would take ~18min, now it's taking around 29min. Is an overhead of this magnitude to be expected?
Thanks
Kind regards,

Hi,I run: python pyscripts/inference/inference.py
The following error message appears:
Traceback (most recent call last):
File "pyscripts/inference/inference.py", line 14, in
from seg_models.models.pspnet import pspnet_resnet101 as model
ImportError: No module named seg_models.models.pspnet

What is the problem, please?
thanks!

Hi, I tried your affinity loss (not adaptive) as my loss function, my network is DeeplabV3+, MobileNet, My own dataset. I set margin=3.0, lambda1=1.0, lambda2=1.0
But there is something wrong with the loss, the not-edge loss is really small and not converge.

Here is a part of nor-edge loss value during training:
Mean Aff Loss is:[6.15826357e-05] Mean Aff Loss is:[7.15486458e-05] Mean Aff Loss is:[4.56848611e-05] Mean Aff Loss is:[5.51421945e-05] Mean Aff Loss is:[7.94407606e-05] Mean Aff Loss is:[0.000143873782] Mean Aff Loss is:[6.04316447e-05] Mean Aff Loss is:[9.94381699e-05] Mean Aff Loss is:[0.000107184518] Mean Aff Loss is:[6.87552383e-05] Mean Aff Loss is:[7.98113e-05] Mean Aff Loss is:[0.000122067388] Mean Aff Loss is:[5.42108719e-05]

As for edge loss value, it will alert Nan or Inf in the beginning. It troubles me so much :(

Could anyone give some advice?

Hello!
I am bit confused about your approach. You take the color labels with 21 classes, and convert it to a gray image and then use a binary loss to optimize it, am I correct?

Sorry, I am currently loading the pre-trained model according to the parameter settings in the paper and the sample file named train_pspnet.sh in the bashscripts directory, also using the PASCAL VOC data set, but the same problem still occurs, the loss function can not be reduced, resulting in the prediction result of the verification data set is not correct,which it can't display the content, I would like to ask you how to solve this situation, thank you for your help.

In your paper, the detail for calculating BR is not offered. Also, in your code, I can't find any cue.
You only offer a citation to Contour Detection and Hierarchical Image Segmentation.
But in this paper, the authors refered a way to output the precision-recall curve, instead of a concrete value. So, I just wonder, how to calculate this