zardinality / tf-deformable-conv Goto Github PK

The calculation of padding size seems to be conflicted in the kernel registration and execution. In kernel registration, the padding size is calculated based on the effective kernel size (taking dilation into consideration), while in the execution part, it is calculated based on the real kernel size.

I don't know how much this will affect the performance, but the output result does change given different padding sizes when dilation is not 1 (has been tested).

when i build my own model,in this step:

i meet the errror:

look forward to hearing from you.
@Zardinality

Hello, the original implementation has supported the cudnn 6.0 and cuda 8.0. Could you upgrade your lib to support the cudnn 6.0 and cuda 8.0. Because I am running in Tensorflow 1.4, so it does not support the lower version of cudnn (i.e 5.0 as your current code). Thanks for the great lib
P.s: I have close previous issue in other project due to wrong location

I met a problem that when i set padding="VALID", the loss function did not converge. However, it converged when i set padding="SAME"...other config is same.
And i also dumped the offset. it was clipped by 0？？ （I didn't use tf.clip at all)

Hi I tried to reimplement the similar operation as yours in Tensorflow and found two facts w.r.t the runtime speed performance:

1. The gpu kernel which adds the filter gradients from one batch to another has almost no influence on the speed performance, in fact, the original MXNet implementation also applies this idea.

2. Splitting back propagations for different inputs variables into different TF ops does help to accelerate the runtime speed, but there's only 30% boost observed, compared with wrapping them into one TF op.

I think the straggler is most likely to be the im2col/col2im operation, which is implemented in pure cuda code with little optimizations (compared with CuDNN). And the Author of Deform Conv also admitted that the main downside of their implementation is that they did not apply any CuDNN for the optimization (sorry I cannot find the origin).

Hopefully, these results can be helpful for those who are also interested in the Deform Conv implementation in Tensorflow, especially when the Deform Conv V2 paper has been released recently.

Any comments or further discussion are welcomed and Merry Christmas!

Hi,

Thanks for your work! I am wondering if you can implement the 3d version as well. It would be very helpful!

This is not an issue because the code worked fine in TF 1.2 and cudnn 5.1
In this question, I want to ask about how can I convert the mxnet code using your implementation. As shown in the line 678, we have

res5a_branch2b_offset_weight = mx.symbol.Variable('res5a_branch2b_offset_weight', lr_mult=1.0)
res5a_branch2b_offset_bias = mx.symbol.Variable('res5a_branch2b_offset_bias', lr_mult=2.0)
res5a_branch2b_offset = mx.symbol.Convolution(name='res5a_branch2b_offset', data = res5a_branch2a_relu,  num_filter=18, pad=(1, 1), kernel=(3, 3), stride=(1, 1),weight=res5a_branch2b_offset_weight, bias=res5a_branch2b_offset_bias)
res5a_branch2b = mx.contrib.symbol.DeformableConvolution(name='res5a_branch2b', data=res5a_branch2a_relu, offset=res5a_branch2b_offset,num_filter=512, pad=(2, 2), kernel=(3, 3), num_deformable_group=1, stride=(1, 1), dilate=(2, 2), no_bias=True)

How can I convert 4 above lines using deform_conv_op.deform_conv_op? I read the demo.py, test_deform_conv.py, and this is my current coverting

import tensorflow.contrib.layers as ly
from lib.deform_conv_op import deform_conv_op
res5a_branch2b_offset = ly.conv2d(res5a_branch2a_relu, num_outputs=18, kernel_size=3,  stride=2, activation_fn=None, data_format='NHWC')
num_x = res5a_branch2a_relu.shape[self.channel_axis].value
res5a_branch2b_kernel= tf.get_variable('weights', shape=[3, 3, num_x, 512])
res5a_branch2b = deform_conv_op(res5a_branch2a_relu, filter=o_b2b_kernel, offset=o_b2b_offset,
                                        rates=[1, 2, 2, 1], padding="SAME", strides=[1, 1, 1, 1],
                                        num_groups=1, deformable_group=1, name='%s/bottleneck_v1/conv2' % name)

Note that, above converting used NHWC order and still missing two first lines

res5a_branch2b_offset_weight = mx.symbol.Variable('res5a_branch2b_offset_weight', lr_mult=1.0)
res5a_branch2b_offset_bias = mx.symbol.Variable('res5a_branch2b_offset_bias', lr_mult=2.0)

And also got the error

ValueError: Deformconv requires the offset compatible with filter, but got: [4,64,64,18] for 'resnet_v1_101/block4/unit_1/bottleneck_v1/conv2' (op: 'DeformConvOp') with input shapes: [4,64,64,512], [3,3,512,512], [4,64,64,18].

face a problem when ./g++_complie.sh
（I use anaconda and i have setted the tensorflow dir）

the error information is as below：

deform_conv.cc:24:61: fatal error: third_party/eigen3/unsupported/Eigen/CXX11/Tensor: No such file or directory
#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor"
^
compilation terminated.

how can i fix it？Thanks！

when I dio the sh ./lib/nvcc_compile.sh operations, I meet the folllowing errors, do you have any idea about how to solve it?

/home/tensor-server/.pyenv/versions/anaconda2-5.0.1/envs/df/lib/python2.7/site-packages/tensorflow/include
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
gcc: error: deform_conv.cu.cc: No such file or directory
gcc: warning: ‘-x c++’ after last input file has no effect
gcc: fatal error: no input files
compilation terminated.

Compilation with gcc6+ failed. Could you please help?

hi,both variables dx and dy range from -2 to 2,can u offer some suggestion why these two variable should change from -2 to 2?

I used it in tensorflow1.4, and met the bug,
tensorflow.python.framework.errors_impl.NotFoundError: /home/master/Deform/lib/deform_conv.so: undefined symbol: _ZTIN10tensorflow8OpKernelE
and my g++ is 4.9.4.
nvcc_compile.sh is as below:
nvcc -std=c++11 -arch=sm_37 -ccbin=/usr/bin/g++-4.9 -c -o deform_conv.cu.o deform_conv.cu.cc -I $TF_INC -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -L /usr/local/cuda-8.0/lib64/ --expt-relaxed-constexpr

g++_compile.sh is as below:
g++-4.9 -std=c++11 -shared -o deform_conv.so deform_conv.cc deform_conv.cu.o -I $TF_INC -fPIC -lcudart -L $CUDA_HOME/lib64 -D GOOGLE_CUDA=1 -Wfatal-errors -I $CUDA_HOME/include -D_GLIBCXX_USE_CXX11_ABI=0

thanks~

Thanks for your amazing work. Have you ever tried to visualize the deformable convolution just like the results in the paper 'Deformable Convolutional Networks'? like this:

I made a try to visualize your deformable convolution code. But I get the following results:


Can you help me?

is it the correct version of deformable conv? and as for the training details, do you have some advices? for examples,learning rate？

when compiled with g++:
/python3.4/site-packages/tensorflow/include/tensorflow/stream_executor/dso_loader.h:32:30: fatal error: cuda/cuda_config.h: No such file or directory compilation terminated.

solved by copy a cuda_config.h file from https://insight.io/github.com/tensorflow/tensorflow/blob/master/third_party/toolchains/gpus/cuda/cuda/cuda_config.h?line

the loss has gotten diverged and the values of offset are very large. So i don't understand the meaning of the current offset.. do you have any idea?