deepsense-ai / roi-pooling

RoI pooling in TensorFlow

This repo contains the implementation of Region of Interest pooling as a custom TensorFlow operation. The CUDA code responsible for the computations was largely taken from the original Caffe implementation by Ross Girshick.

For more information about RoI pooling you can check out Region of interest pooling explained at our deepsense.io blog.

Requirements

To compile and use roi_pooling layer you need to have:

• CUDA (tested with 8.0)
• https://www.tensorflow.org/ (tested with 0.12.0 and 1.0.0)

Only official TensorFlow releases are currently supported. If you're using a custom built TensorFlow compiled with a different GCC version (e.g. 5.X) you may need to modify the makefile to enable the new ABI version.

Install

Since it uses compilation

$ git clone [email protected]:deepsense-io/roi-pooling.git
$ cd roi-pooling
$ python setup.py install

Right now we provide only GPU implementation (no CPU at this time).

Usage

After successful installation you can use the operation like this:

from roi_pooling.roi_pooling_ops import roi_pooling

# here obtain feature map and regions of interest
rpooling = roi_pooling(feature_map, rois, 7, 7)
# continue the model

Working example in Jupyter Notebook: examples/roi_pooling_minimal_example.ipynb

roi-pooling's Issues

can't install

I'm trying to install, but found these error. what did I miss?

make: Entering directory `E:/Tugas/Semester 7/Code/fasterrcnn/roi-pooling-master/roi_pooling'
g++ -std=c++11 -shared -o roi_pooling.so roi_pooling.cc roi_pooling.cu.o -I C:\Users\USER\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\include -fPIC -lcudart -L/usr/local/cuda-8.0/targets/x86_64-linux/lib/ -D _GLIBCXX_USE_CXX11_ABI=0
process_begin: CreateProcess(NULL, g++ -std=c++11 -shared -o roi_pooling.so roi_pooling.cc roi_pooling.cu.o -I C:\Users\USER\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\include -fPIC -lcudart -L/usr/local/cuda-8.0/targets/x86_64-linux/lib/ -D _GLIBCXX_USE_CXX11_ABI=0, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [roi_pooling.so] Error 2
make: Leaving directory `E:/Tugas/Semester 7/Code/fasterrcnn/roi-pooling-master/roi_pooling'
running build
running build_py
running install_lib
running install_egg_info
Removing C:\Users\USER\AppData\Local\Programs\Python\Python35\Lib\site-packages\roi_pooling-1.0-py3.5.egg-info
Writing C:\Users\USER\AppData\Local\Programs\Python\Python35\Lib\site-packages\roi_pooling-1.0-py3.5.egg-info

regarding the compiling issue

Hi,

Thanks for sharing the code.

Because I am working on a public computer node, and have some difficulties of running python setup install,

running install_lib
creating /tmp/lib/python2.7/site-packages/roi_pooling
error: could not create '/tmp/lib/python2.7/site-packages/roi_pooling': Permission denied

As a result, when running the test code like from roi_pooling.roi_pooling_ops import roi_pooling, it causes the following error message

Traceback (most recent call last):
  File "test_run.py", line 7, in <module>
    from roi_pooling.roi_pooling_ops import roi_pooling
  File "/home/DL-Phase3/roi-pooling-master/roi_pooling/roi_pooling_ops.py", line 8, in <module>
    roi_pooling_module = tf.load_op_library(lib_path)
  File "/tmp/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
    None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/DL-Phase3/roi-pooling-master/roi_pooling/roi_pooling.so: cannot open shared object file: No such file or directory

Are there any workaround? Thanks.

Running the setup.py

I have cloned the roi-pooling and i am trying to install the setup.py using the install command as given but i am getting error:

.....from._conv import register_converters as _registers_converters running install
error: [winerror 2]the system cannot find the file specified

Unable to reproduce minimal_example on tensorflow 1.3

I'm getting all zeros when I execute the minimal example Notebook. I'm using tensorflow 1.3 and python 3

[[[[ 0. 0.]
[ 0. 0.]]]

[[[ 0. 0.]
[ 0. 0.]]]

[[[ 0. 0.]
[ 0. 0.]]]]
Edit: Problem with my tensorflow

I'm very confused with ROI pooling's result.

Hi,thank you for your great work.I have a problem.
If
input_value = [[
[[1], [2], [4], [4],[5],[6]],
[[3], [4], [1], [12],[5],[6]],
[[6], [2], [1], [7.0],[5],[6]],
[[1], [0], [2], [8],[5],[6]]
]]

rois_value = [
[0, 0, 0, 3, 5] #4*6
]

y = roi_pooling(input, rois, pool_height=2, pool_width=2)

then,the output is
y: [[[[ 6. 12.]
[ 1. 8.]]]]

I think a sub-window’s size is 2*3,so the result should be
[[[[ 4. 12.]
[ 6. 8.]]]]
So can you tell me how your code works?Thank you.

Backprop through the RoI layer

Hi ,

Thanks for providing the roi pool layer in tensorflow. I have a question regarding the backprop. As we know all the ops in tensorflow are differentiable. I mean if I add an op in my graph tensorflow will be able to backprop through that op during the optimization process. Is your implementation of roi pool differentiable ?I mean if I add this in my graph do I need to worry about implementing the back propagation of gradients through this op?

Thanks

Error when compiled with gcc 5

First off, a big thank you for making/open-sourcing this!

I had used the roi-pooling op before, but decided to go from the TF pip package to compiling it myself from source, which worked fine, but when I recompiled the roi-pooling, I got an error "undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv"

Google led me to GPflow/GPflow#268 and in turn GPflow/GPflow#227

So after a bit of looking around the source, I saw the "-D _GLIBCXX_USE_CXX11_ABI=0" part in the roi_pooling/Makefile (lines 9 and 12) and after removing that, everything compiles and works as before.

Maybe you could say so in the README or adjust the Makefile to work with gcc4 and 5?

Thought I'd let you (and other future users) know, in case anyone else runs into this problem

Thanks again for a great plugin!

`module 'tensorflow' has no attribute 'sysconfig'` on running `make`

Getting the following error, module 'tensorflow' has no attribute 'sysconfig'
on running make, as below
$ make
rm -f *.o *.so *.pyc *.npy
Traceback (most recent call last):
File "<string>", line 1, in <module>
AttributeError: module 'tensorflow' has no attribute 'sysconfig'
nvcc -std=c++11 -c -o roi_pooling.cu.o roi_pooling.cu.cc -I -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -D _GLIBCXX_USE_CXX11_ABI=0
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
make: *** [Makefile:9: roi_pooling.cu.o] Error 1

why no atomic protection in backward?

The code here https://github.com/deepsense-ai/roi-pooling/blob/d1ae7e0b702d1f4a1672e793184324379b96ecb0/roi_pooling/roi_pooling.cu.cc#L196 didn't use any protection for the racing between different threads. Why we don't need this? There is possible that the same pixel been used in different output positions, so different threads may add gradient to the same places of the gradient output.
Thanks.

License for this project

Hi Tomasz!

Great work, thank you so much for the implementation. Can you please let me know the license you wish to release the project under? Thank you very much in advance.

Best regards,
Arjun
https://www.cse.iitb.ac.in/~ajain/

import rio-pooling problem

I run "python setup.py install" successfully , then when I import roi-pooling with
"from roi_pooling.roi_pooling_ops import roi_pooling" , I get a problem:

from roi_pooling.roi_pooling_ops import roi_pooling
Traceback (most recent call last):
File "", line 1, in
File "roi_pooling/roi_pooling_ops.py", line 8, in
roi_pooling_module = tf.load_op_library(lib_path)
File "/home/forever/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
File "/home/forever/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: /home/forever/anaconda2/envs/tensorflow/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/forever/roi-pooling/roi_pooling/roi_pooling.so)

How can I fix it?

Issues related to non existence of a file in this repo

I have written my own code for doing RoI pooling of an image.
Here is the code :
from roi_pooling.roi_pooling_ops import roi_pooling
from keras.applications.vgg16 import VGG16
from keras.models import Model
import matplotlib.pyplot as plt
import cv2
import tensorflow as tf

img=cv2.imread("img_1.jpg")
model = VGG16()

feature_map=model.predict(img)

feature_map = np.asarray(feature_map, dtype='float32')

rois_value = [ [0, 0, 0, 1, 3], [0, 2, 2, 3, 3], [0, 1, 0, 3, 2] ]

rois= np.asarray(rois_value, dtype='int32')

input_const = tf.constant(feature_map, tf.float32) rois_const = tf.constant(rois, tf.int32)

rpooling = roi_pooling(input_const, rois_const, 7, 7)

plt.imsave("1.png",rpooling)
`

My directory looks like this:
->roi_pooling
----->roi_pooling_test.py
----->roi_pooling_ops.py
----->roi_pooling.cu.o
----->roi_pooling.cu.cc
----->test.py
->setup.py
->build
----->lib
-------->roi_pooling
----------->roi_pooling_test.py
----------->roi_pooling_ops.py
->image.jpg (testing image)

The error shown is
Traceback (most recent call last): File "roi.py", line 2, in <module> from roi_pooling.roi_pooling_ops import roi_pooling File "/home/pg2017/cse/17071016/.conda/envs/myenv/lib/python3.7/site-packages/roi_pooling/roi_pooling_ops.py", line 8, in <module> roi_pooling_module = tf.load_op_library(lib_path) File "/home/pg2017/cse/17071016/.conda/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/pg2017/cse/17071016/.conda/envs/myenv/lib/python3.7/site-packages/roi_pooling/roi_pooling.so: cannot open shared object file: No such file or directory

Can anybody address this problem?

Thanks in advance. Waiting for a reply.

NOTE
I have tensorflow version 1.13.0 and it can't be downgraded to 1.0.0

How to generate the roi_pooling.cu.o? Thanks for help

I got the following errors:

make: nvcc: Command not found
Makefile:9: recipe for target 'roi_pooling.cu.o' failed
make: *** [roi_pooling.cu.o] Error 127

The third ROI coordinate comment is not correct

In "roi_pooling_minimal_example.ipynb":

# * channel 0, rectangular region (1, 0) to (3, 2)
#              ....
#              xxx.
#              xxx.
#              xxx.

I think this comment should be:

# * channel 0, rectangular region (1, 0) to (3, 2)
#              .xxx
#              .xxx
#              .xxx
#              ....

NotFoundError: roi_pooling.so

Hey,
I have tried to build it but got the error
"NotFoundError: /opt/anaconda/lib/python3.6/site-packages/roi_pooling/roi_pooling.so: cannot open shared object file: No such file or directory"

Can someone help me? Is this a problem with anaconda?

Undefined symbol on tensorflow 1.8

I am getting an error on roi_pooling.so: undefined symbol. on tensorflow 1.8
Which version is best suitable for roi pooling ?

I wonder roi_pooling_minimal_example.ipynb

Update to CUDA-10

Library no longer functions when upgraded to CUDA-10, Tensorflow 1.13.

Exact error: once upgraded, tests fail with

self.assertEqual(y.get_shape().ndims, 4)
AssertionError: None != 4

It seems that the output tensor always has shape [unknown].

New makefile for Cuda10

TF_INC = $(shell python3 -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
TF_CFLAGS = $(shell python3 -c "import tensorflow as tf; print(' '.join(tf.sysconfig.get_compile_flags()))")
TF_LFLAGS = $(shell python3 -c "import tensorflow as tf; print(' '.join(tf.sysconfig.get_link_flags()))")

CUDA_LIB = /usr/local/cuda-10.0/targets/x86_64-linux/lib/

all: clean build test

build: roi_pooling.so

roi_pooling.cu.o: roi_pooling.cu.cc
	nvcc -std=c++11 -c -o $@ $? -I $(TF_INC) -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

roi_pooling.so: roi_pooling.cc roi_pooling.cu.o
	g++ -std=c++11 -shared -o $@ $? $(TF_CFLAGS) $(TF_LFLAGS) -fPIC -L$(CUDA_LIB) -O2

test: build
	python3 roi_pooling_test.py

clean:
	rm -f *.o *.so *.pyc *.npy

Steps to build for Cuda10...

1. Have Tensorflow 1.13 and Cuda 10 working.
2. Update makefile (seen above)
3. Follow previous install instructions.

I want to use average pooling rather than max pooling in roi pooling

I want to use average pooling rather than max pooling in roi pooling.

However, the roi-pooling you've implemented is customized op, and I am confused and not sure what to change.

May I ask you how I should change code to use average pooling in roi-pooling module, or give me a hint?

Running it with tensorflow 1.3.0

I receive following error when I build it with tensorflow 1.3.0.

Makefile:9: recipe for target 'roi_pooling.cu.o' failed

any idea why it happens like this. ?

Problem running with tensorflow 1.3

I have my server installed tensorflow and successfully compiled and installed roi-pooling
however when I try to run the minimal demo, I get the following error:
Anyone know what's going on here?

InvalidArgumentError: No OpKernel was registered to support Op 'RoiPooling' with these attrs. Registered devices: [CPU], Registered kernels:
device='GPU'

[[Node: RoiPooling = RoiPooling[pool_height=2, pool_width=2](Const, Const_1)]]

Caused by op 'RoiPooling', defined at:
File "/home/meta/anaconda3/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/meta/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/home/meta/anaconda3/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "/home/meta/anaconda3/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/meta/anaconda3/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/meta/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/meta/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/meta/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/meta/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/meta/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/meta/anaconda3/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/meta/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2717, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/meta/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2821, in run_ast_nodes
if self.run_code(code, result):
File "/home/meta/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 5, in
y = roi_pooling(input_const, rois_const, pool_height=2, pool_width=2)
File "/home/meta/anaconda3/lib/python3.6/site-packages/roi_pooling/roi_pooling_ops.py", line 19, in roi_pooling
out = roi_pooling_module.roi_pooling(input, rois, pool_height=pool_height, pool_width=pool_width)
File "", line 40, in roi_pooling
File "/home/meta/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/meta/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/meta/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): No OpKernel was registered to support Op 'RoiPooling' with these attrs. Registered devices: [CPU], Registered kernels:
device='GPU'

[[Node: RoiPooling = RoiPooling[pool_height=2, pool_width=2](Const, Const_1)]]

import issue

When I run "from roi_pooling.roi_pooling_ops import roi_pooling" in python with tensorflow-gpu, I got a error:

from roi_pooling.roi_pooling_ops import roi_pooling
Traceback (most recent call last):
File "", line 1, in
File "roi_pooling/roi_pooling_ops.py", line 8, in
roi_pooling_module = tf.load_op_library(lib_path)
File "/home/forever/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
File "/home/forever/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: /home/forever/roi-pooling/roi_pooling/roi_pooling.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

How can I fix it?THANKS.

Import roi_pooling,get error below

"/lib/python2.7/site-packages/roi_pooling/roi_pooling.so: cannot open shared object file: No such file or directory"

Compiling on Windows 10

Hi,

Thanks for sharing the code.

I'm trying to use your api in my project. But it seems that your compilation instruction can only be run on a Linux system. Since I am using tensorflow on a windows system, some errors about missing roi_pooling.so appeared. It would be kind of you if you can provide some instructions on how to compile them on windows 10. Thanks.

Does roi_pooling op support multi-input?

if the input is (batch_size x height x width x channel) , what's the corresponding roi shape should be?