fabianbormann / tensorflow-deconvnet-segmentation

Tensorflow implementation of "Learning Deconvolution Network for Semantic Segmentation"

Tensorflow implementation of Learning Deconvolution Network for Semantic Segmentation.

Install Instructions

1. Works with tensorflow 1.11.0 and uses the Keras API so use pip to install tensorflow-gpu in the latest version

2. Run the following commands in your terminal

git clone https://github.com/fabianbormann/Tensorflow-DeconvNet-Segmentation.git
cd Tensorflow-DeconvNet-Segmentation
sudo pip3 install -r requirements.txt

python3

Python 3.5.2+ (default, Sep 22 2016, 12:18:14)
[GCC 6.2.0 20160927] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from DeconvNet import DeconvNet
>>> deconvNet = DeconvNet() # will start collecting the VOC2012 data
>>> deconvNet.train(epochs=20, steps_per_epoch=500, batch_size=64)
>>> deconvNet.save()
>>> prediction = deconvNet.predict(any_image)

Contributions welcome!

tensorflow-deconvnet-segmentation's Issues

Using activation function only in conv layer?

Thanks for sharing good stuff!
Reviewing this code, and found that in file DeconvNet.py ReLU used only once in convolution layer, not in deconvolution layer. In ref paper they are using. Is this correct?
(For me, looks like problem arrives from calc of output score values that don't need activation function, linear op, but deconv need relu..)

batch normalization, and input size

Hi!
I wonder why batch normalization and activation function is not used in the code.
Also, another thing that I am curious about is that filter size of 'fc_6' is 77 in this code. however , the filter width&height should be variable to make fc_7 11 vector.

Thank you.

Weights getting initialized every forward pass?

Hi,

I have a slight confusion here, every time the conv_layer or deconv_layer function is called, then the weights and biases are being initialized. But then will this happen every forward pass? If not how are the weights of each conv and deconv layer getting saved, i mean how to access those variables?

bug

Traceback (most recent call last):
File "img_to_tfrecords.py", line 48, in
iQ,imgLen,mQ,segLen=read_filelist(args.imgpath,args.segpath)
File "img_to_tfrecords.py", line 20, in read_filelist
imgFiles=tf.train.string_input_producer(imgList,shuffle=False)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 217, in string_input_producer
raise ValueError(not_null_err)
ValueError: string_input_producer requires a non-null input tensor

Requesting Documentation for unpool_layer2x2 in DeconvNet.py

I know this question should not be an issue, but I would like to read some documentation on the unpool_layer2x2 method in DeconvNet.py code so as to understand what is going on in this code.

Any help is much appreciated!!

Prediction

Hi Fabian!

Thanks for sharing this! Have you completed the prediction component yet?

Thanks in advance!
Lilly

Error in DeconvNet.py

There are some errors I've found recently when trying to reproduce the result.

1. restore_session -> doesn't have 'self' as param
2. in predict: feed_dict={image:[image]} -> {self.x:[image]}

Logic behind MaxUnpoolWithArgmax

I have been reading thorugh the code for the function "MaxUnpoolWithArgmax" since I'm interested in the logic to extract the indexes [b,y,x,c] from the argmax tensor. I cannot understand the maths behind to extract the index "y" and "x". Could you elaborate a bit on how did you get these expressions? :

• y = argmax // (output_shape[2] * output_shape[3])

• x = argmax % (output_shape[2] * output_shape[3]) // output_shape[3]

Thank you so much
Pedro

How to predict?

I used the code:
'''image = np.float32(cv2.imread(image_path))
deconvNet = DeconvNet()
deconvNet.restore_session()
deconvNet.predict(image=image)'''
But it always doesn't work . . .(Help Me...)

respect batch_size > 1 in unpooling/unravel method

How can I train for 2 classes?

I want to use the same dataset (pascal voc) for training, but change the number of classes.
I only want 2 classes, background (black) and my object (white)...it's a binary segmentation.

thanks

logits' dimension

Hi?
I am currently working to make PR of DeconvNetPipeline.py with batch normalization.
I wonder, however, why the logits' last dimension is 21 at the end of the network, while ?
Does self.y has same 21 at final dimension?

Thank you

i got enormous loss function

hi i run the code DeconvNetPipeline.py and while the traning i got something like this:
2017-01-26 15:01:00.156146: step 85, loss = 39709622646341632.00 (3.6 examples/sec; 2.809 sec/batch) 2017-01-26 15:01:03.019399: step 86, loss = 34950307108618240.00 (3.5 examples/sec; 2.863 sec/batch) 2017-01-26 15:01:05.870122: step 87, loss = 37934860555255808.00 (3.5 examples/sec; 2.851 sec/batch)
its error?
Also he break in the middle and because the line:
except tf.errors.OutOfRangeError:
what its mean?
thenk you

[Suggestion] Rewrite the download script in python

Makes it multi platform by default

Shape mismatch while unpooling

The Error appears in line 147 respectively in line 234. Maybe there is some flooring or ceiling during the input size and or conv layers.

Steps to reproduce:

cd /your/path/to/Tensorflow-DeconvNet-Segmentation/
python3 main.py

Error: Caused by op 'concat_1', defined at:
File "main.py", line 4, in
deconvNet = DeconvNet(use_cpu=True)
File "/home/fabian/workspace/Tensorflow-DeconvNet-Segmentation/DeconvNet.py", line 15, in init
self.build(use_cpu=use_cpu)
File "/home/fabian/workspace/Tensorflow-DeconvNet-Segmentation/DeconvNet.py", line 147, in build
unpool_4 = self.unpool_layer2x2(deconv_5_1, self.unravel_argmax(pool_4_argmax, tf.to_int64(tf.shape(conv_4_3))))
File "/home/fabian/workspace/Tensorflow-DeconvNet-Segmentation/DeconvNet.py", line 234, in unpool_layer2x2
t = tf.concat(3, [t2, t1])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 872, in concat
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 436, in _concat
values=values, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2380, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1298, in init
self._traceback = _extract_stack()

How to deal with images with various size in VOC

Hi, I have saw that in this code you use the placeholder with fixed shape? I really want to know
how to deal with images with different size in semantic segmentation.

Unpooling gradient problem

I try to use a your unpooling implementation. thank you for sharing though.
It works greatly during the forward pass.
but, when back-propagating the gradient through the unpooling layer, i think they don't.
i checked parameters of child(bottom) layer of unpooling layer at training time, they are not changed at all.
do you know why this problem happen?

About input's shape

I am wondering whether the shape of this input must be (224, 224, 3). If it is not this shape, I just need to change the kernel_size in fc6 in the code to the width and height of pool5, right?

fc6 = Conv2D(512, 7, use_bias=False, padding='valid', name='fc6')(pool5) #4096
fc6 = BatchNormalization(name='batchnorm_fc6')(fc6)
fc6 = Activation('relu', name='relu_fc6')(fc6)

Trained weights?

Hello! Is the trained network weights available for download somewhere?

Training Speed

Hi Fabian, sorry this might not entirely constitute an "issue", but I was wondering if it is normal to have training times of 1-5 seconds per step? Meaning that 1000 steps of GD might take ~1 hour? This seems quite slow based on other networks I've trained (on my GTX 1080 GPU), particularly with a single image per batch, but perhaps this is entirely down to the significant number of parameters for this very deep model. Just wanted to make sure it seemed like everything was installed and running properly on my machine anyway as a sanity check! Thanks!

An error message while running the UnpoolLayerTest.ipynb

Hi Fabian,

When experimenting with your code of UnpoolLayerTest.ipynb, I got the following error message. Would you like to take a look at it? I did not change your code. Thanks.

Traceback (most recent call last): File "UnpoolLayerTest.py", line 56, in <module> unpool_1 = unpool_layer2x2(pool_1, unravel_argmax(argmax, tf.to_int64(tf.shape(x)))) File "UnpoolLayerTest.py", line 17, in unpool_layer2x2 output = tf.zeros([x_shape[1] * 2, x_shape[2] * 2, x_shape[3]]) File "/develop/tfw/lib/python3.4/site-packages/tensorflow/python/ops/array_ops.py", line 620, in zeros output = constant(0, shape=shape, dtype=dtype, name=name) File "/develop/tfw/lib/python3.4/site-packages/tensorflow/python/ops/constant_op.py", line 162, in constant tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape)) File "/develop/tfw/lib/python3.4/site-packages/tensorflow/python/framework/tensor_util.py", line 363, in make_tensor_proto shape = [int(dim) for dim in shape] File "/develop/tfw/lib/python3.4/site-packages/tensorflow/python/framework/tensor_util.py", line 363, in <listcomp> shape = [int(dim) for dim in shape] TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tensor'

Data url changed

404 error when downloading the data

Tensorflow in Windows

Can I compile and execute this code on windows? I tried like

"pip install -r requirement.txt
Then get error:
Could not open requirements file: [Errno 2] No such file or directory: 'requirement.txt'"

I think I may not using right command. I am very beginner at this.......:(

is there any pre-trained weight ?

Hi, fabianbormann,is there any pre-trained weight that we can use for prediction directly?

batch normalization+ReLu

I have two questions regarding model implementation:

1. Is there any specific reason why batch norm is not implemented?
2. why deconv layers doesn't end with ReLu activation?

Thanks!

Want for the Well trained models~

Hello everyone ! Do you have the Well trained models . I want to see the the effect of the network .
As you all know, the network is training very slowly . . .Thanks ! !

Deconv_1_1

Hello,
I read the original paper and there was no deconv layer with 32 filters, but in the code I saw this:

deconv_1_1 = self.deconv_layer(deconv_1_2, [3, 3, 32, 64], 32, 'deconv_1_1')

I really appreciate if someone helps me with that.

URL not found

Hello,

I'm really interested in this work, and i want to test it out.
But during the download i'm not being able to dowload the data.

Is it something from me? I even checked and the url provided is down.

About Prediction

I'm really sorry about it to make this issue with my problem

my question is this.
checkpoint_directory = "/checkpoints/"
img1 =np.float32(cv2.resize(cv2.imread("./test_data/tabby_cat.png"),(224,224)))
img1 =tf.placeholder(tf.expand_dims(img1, 0))

#img2 = tf.placeholder(tf.float32,shape=(1,224,224,3))
#img2 = {img2:img1}

checkpoint_file=tf.train.latest_checkpoint(checkpoint_directory)
session_conf = tf.ConfigProto(allow_soft_placement=True)
session_conf.gpu_options.allocator_type = 'BFC'
init = tf.initialize_all_variables()

trainset = open('data/stage_1_train_imgset/train.txt').readlines()
random_line = random.choice(trainset)
image_file = random_line.split(' ')[0]
ground_truth_file = random_line.split(' ')[1]
image = np.float32(cv2.imread('data' + image_file))
ground_truth = cv2.imread('data' + ground_truth_file[:-1], cv2.IMREAD_GRAYSCALE)

ground_truth = (ground_truth / 255) * 20

sess = tf.Session(config = session_conf)

with sess.as_default():

#saver =tf.train.import_meta_graph("{}.meta".format(checkpoint_file))
#saver.restore(sess,checkpoint_file)
sess.run(init)
deconv = DeconvNet()
#feed_dict = {deconv.x:img2}

#deconv.predict(image)
    
tensors = [deconv.logits, deconv.prediction]
C = sess.run(tensors,feed_dict ={deconv.x:img1})

I finished the "Training" with Deconv.py then tried to make prediction things.
Is there any wrong things for making prediction?
plz help me.