Coder Social home page Coder Social logo

deformable-convolution-pytorch's People

Contributors

1zb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deformable-convolution-pytorch's Issues

Build error

Hi:
When I ran CC=g++ python build.py, I got errors of cffi.error.VerificationError: CompileError: command 'g++' failed with exit status 1. Could you give me some suggestions? Thank you. @1zb
The error details are as follows:

Traceback (most recent call last):
  File "/home/sean/anaconda3/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/home/sean/anaconda3/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/sean/anaconda3/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/sean/anaconda3/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'g++' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/home/sean/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/sean/anaconda3/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/sean/anaconda3/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'g++' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "build.py", line 36, in <module>
    ffi.build()
  File "/home/sean/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 189, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 111, in _build_extension
    outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'g++' failed with exit status 1

make.sh error

run nvcc -c -o deform_conv_cuda_kernel.cu.o deform_conv_cuda_kernel.cu -x cu -Xcompiler -fPIC -std=c++11 happens error,nvcc fatal : Cannot find compiler 'cl.exe' in PATH

test error !

感谢您开源代码 但是test出问题了 能麻烦您给一个类似mnist的测试例子吗

sun@sunwin:~/0newcodedown/deformable-convolution-pytorch-master$ python test.py
Traceback (most recent call last):
File "test.py", line 6, in
from modules import ConvOffset2d
File "/home/sun/0newcodedown/deformable-convolution-pytorch-master/modules/init.py", line 1, in
from .deform_conv import ConvOffset2d
File "/home/sun/0newcodedown/deformable-convolution-pytorch-master/modules/deform_conv.py", line 7, in
from functions import conv_offset2d
File "/home/sun/0newcodedown/deformable-convolution-pytorch-master/functions/init.py", line 1, in
from .deform_conv import conv_offset2d
File "/home/sun/0newcodedown/deformable-convolution-pytorch-master/functions/deform_conv.py", line 5, in
from _ext import deform_conv
ImportError: No module named _ext

offset size

Thanks for your code firstly.
In this implementation, the spatial resolution of offset should be the same with output feature map.
But in the original paper, the spatial resolution of offset is the same with input feature map.
Does this have an effect on the performance?

each kernel shares the same offsets?

#According to test.py, the line 'offset=conv(inputs)' is to learn the offset of neighbor domain of each pixel, so the size of offsets is 118H*W. if the size of input is 1x3xHxW and the size of output is 1x4xHxW, does it means that there has 4 kernels,and each kernel shares the same offsets?

offset number?

why offset numbers are kW x kH x 2?
In the paper ,offset= 2 x N, the N is number of channel。

Error occurred

When the size of input image is larger than 1024, below error is occurred.

error in deformable_im2col: invalid argument

This error need to be fixed.

Potential Memory Leak

First, thanks for the great work.
I found there's possible memory leak in the code.
To reproduce the problem, change the test.py as below:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from torch.autograd import gradcheck
from torch.nn.modules.utils import _single, _pair

from modules import ConvOffset2d

num_deformable_group = 1

N, inC, inH, inW = 1, 3, 512, 512
outC, outH, outW = 4, 512, 512
kH, kW = 3, 3

conv = nn.Conv2d(inC, num_deformable_group * 2 * kH * kW, kernel_size=(kH, kW), stride=(1,1), padding=(1,1), bias=False).cuda()
conv_offset2d = ConvOffset2d(inC, outC, (kH, kW), stride=1, padding=1).cuda()

for epoch in range(500):
    inputs = Variable(torch.randn(N, inC, inH, inW).cuda())
    offset = conv(inputs)
    output = conv_offset2d(inputs, offset)
    output.backward(output.data)
    print(output.size())

Code will crash for OOM while doing the backward()

Is it possible to support 3D convolution?

I want to use 3D version of deformable convolution, but it seems like your code only support 2D. What change should be made to adapt to 3D convolution? Is it possible to just change the number of offsets?

梯度测试不通过

  • 我使用 gradcheck 提供的梯度测试方法,测试梯度不通过。比较数值梯度和解析梯度,其每一个非零数(非边界位置)的符号,位置等都是一样的,但就是数值差了一点。
  • 另外,在cifar10上简单测试,没有效果,它在使用上有什么要注意的吗,可不可以交流一下。

How does num_deformable_groups work?

First off, thank you for your amazing port, @1zb!

How does num_deformable_groups work? I was wondering if it is splitting the offsets, thus each group of input channels will have a given set of offsets (which may be different from other groups). For instance, if we have an input of 6 channels (1, 6, H, W), and num_deformable_groups=2, then we would have our input split between two tensors of (1, 3, H, W) and (1, 3, H, W), each with its respective instance of offset mask (1, 3, kH, kW). Is that correct?

Problem in test.py

I ran the sh make.sh and then CC=g++ python build.py. It seems that It is building fine, however after running the test.py I have got error. Please note that in the _ext/deform_conv I have only an init.py and _deform_conv.
I really appreciate your help.

sh make.sh
CC=g++ python build.py
Including CUDA code.
/home/mika/wd/deformable-convolution-pytorch-master
generating /tmp/tmp_sneiacp/_deform_conv.c
setting the current directory to '/tmp/tmp_sneiacp'
running build_ext
building '_deform_conv' extension
creating home
creating home/mika
creating home/mika/wd
creating home/mika/wd/deformable-convolution-pytorch-master
creating home/mika/wd/deformable-convolution-pytorch-master/src
g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/v/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c _deform_conv.c -o ./_deform_conv.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.c -o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.c -o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -shared -B /home/mika/.conda/envs/py36/compiler_compat -L/home/mika/.conda/envs/py36/lib -Wl,-rpath=/home/mika/.conda/envs/py36/lib -Wl,--no-as-needed -Wl,--sysroot=/ ./_deform_conv.o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.o /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda_kernel.cu.o -o ./_deform_conv.so

python test.py
Traceback (most recent call last):
File "test.py", line 6, in
from modules import ConvOffset2d
File "/home/v/wd/deformable-convolution-pytorch-master/modules/init.py", line 1, in
from .deform_conv import ConvOffset2d
File "/home/mika/wd/deformable-convolution-pytorch-master/modules/deform_conv.py", line 7, in
from functions import conv_offset2d
File "/home/mika/wd/deformable-convolution-pytorch-master/functions/init.py", line 1, in
from .deform_conv import conv_offset2d
File "/home/mika/wd/deformable-convolution-pytorch-master/functions/deform_conv.py", line 5, in
from _ext import deform_conv
File "/home/mika/wd/deformable-convolution-pytorch-master/_ext/deform_conv/init.py", line 3, in
from ._deform_conv import lib as _lib, ffi as _ffi
ImportError: /home/mika/wd/deformable-convolution-pytorch-master/_ext/deform_conv/_deform_conv.so: undefined symbol: __cudaPopCallConfiguration

performance issue

I have the following network:

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1,8,3,padding=1,bias=False)
        self.batch1 = nn.BatchNorm2d(8,affine=False)
        #self.conv2 = nn.Conv2d(8,16,3,padding=0,bias=False)
        self.conv2offset = nn.Conv2d(8,2*3*3,3,padding=0,bias=False)
        self.deform_conv2 = ConvOffset2d(8,16,3,padding=0,num_deformable_groups=1)
        self.batch2 = nn.BatchNorm2d(16,affine=False)
        self.pooling = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(6*6*16,10)
        self.activation = nn.ReLU()

    def forward(self,x):
        x = self.conv1(x)
        x = self.pooling(x)
        x = self.batch1(x)
        x = self.activation(x)
        #x = self.conv2(x)
        offset = self.conv2offset(x)
        x = self.deform_conv2(x,offset)
        x = self.pooling(x)
        x = self.batch2(x)
        x = self.activation(x)
        logits = self.fc1(x.view(-1,6*6*16))
        probas = F.softmax(logits, dim=1)
        return logits, probas

I train it on MNIST for 2 batches. It takes 327 seconds to run (97.64% accuracy on test set).

Now if I remove deform conv and replace it with normal convolution (commented in the code above), it takes 19 seconds for 2 batches (97.54% accuracy on test set).

What do you think is the cause?

Pytorch v0.3.0
Python v3.6.1

Different interpretation of def_conv:

Thinking about deformable convolutions, some things I found different interpretations of:
-) Does the offset change for each individual k x k kernel or is it fixed for the whole image? Would this mean that pixels could potentially overlap?
-) Is the same offset then applied for each input layer, ie. AxBxC where C might be any number of filters.
-) During inference, keeping the offset generating layers in the network, each k x k kernel would experience an individual offset, or would the offset be the same for the whole image?

Extend this to 1-D

I'm trying to extend ConvOffset2d to 1d convolution as well.

While this can be easily done by simply consider a length-L 1D tensor as a dimension [1xL] 2D tensor, the problem is that the offset now is for 2D deformation (you can go up or down, left or right); whereas in 1D, I only want the kernel connection to move by left/right.

I'm seeking to change the .c and .cu code so that I no longer need to worry about the up/down offset, and in offset = Conv2d(...) I only need output dimension of num_deformable_groups * kernel_size, instead of num_deformable_groups * 2 * kernel_size`.

Any suggestion on which parts of the .c and .cu that I should change? Thank :-)

License

Hi, @1zb, Thanks for your great work! Can you add a license to this repository such that this code can be used in our own projects?

Pytorch的C扩展相关资料

请问哪里有Pytorch的C扩展相关文档或样例,官方给的GitHub例子太简单了,你是直接根据那个写出来的吗?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.