johnqczhang / densepose_installation Goto Github PK

undefined symbol: _ZN6caffe219CPUOperatiorRegistryB5cxx11Ev

Hi,

First of all thank you very much for this, we have been struggling a lot to install densepose in rhel 7 whose environment is like centos 7.

Most all of the procedure went smoothly (although I had to favor a caffe2 building form source instead of going through the conda binary version) except that for the step where you wrote:
"Download the source code of PyTorch, then copy the folder $PYTORCH/caffe2/utils/threadpool into $TORCH_PATH/lib/include/caffe2/utils/".

In fact, I also had to do the same but with a math folder:
Copy the folder $PYTORCH/caffe2/utils/math into $TORCH_PATH/lib/include/caffe2/utils/.

Don't know why, it may be related to recent updates to pytorch. You may add this mentioning that in some cases this step may be necessary.

Best regards.

Sorry to interrupt again, I configured the environment on another computer, always follow your installation instructions, until the last "cd build && cmake .. && make" I recieved this problem:

(DensePose) [zhushiwei@SH-IDC1-10-5-40-31 DensePose]$ cd build/
(DensePose) [zhushiwei@SH-IDC1-10-5-40-31 build]$ cmake ..
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /bin/cc
-- Check for working C compiler: /bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /bin/c++
-- Check for working CXX compiler: /bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "9.0")
-- Caffe2: CUDA detected: 9.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 9.0
-- Found CUDNN: /usr/local/cuda/include/
-- Found cuDNN: v7.0.4 (include: /usr/local/cuda/include/, library: /usr/local/cuda/lib64/libcudnn.so)
-- Autodetected CUDA architecture(s): 6.1 6.1 6.1 6.1
-- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
-- Summary:
-- CMake version : 3.14.0
-- CMake command : /mnt/lustre/zhushiwei/anaconda3/envs/DensePose/bin/cmake
-- System name : Linux
-- C++ compiler : /bin/c++
-- C++ compiler version : 4.8.5
-- CXX flags : -std=c++11 -O2 -fPIC -Wno-narrowing
-- Caffe2 version : 1.1.0
-- Caffe2 include path : /mnt/lustre/zhushiwei/anaconda3/envs/DensePose/lib/python3.7/site-packages/torch/include
-- Caffe2 found CUDA : True
-- CUDA version : 9.0
-- CuDNN version : 7.0.4
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/lustre/zhushiwei/senseTimeWork/DensePose/build
(DensePose) [zhushiwei@SH-IDC1-10-5-40-31 build]$ make
Scanning dependencies of target caffe2_detectron_custom_ops
[ 12%] Building CXX object CMakeFiles/caffe2_detectron_custom_ops.dir/detectron/ops/pool_points_interp.cc.o
[ 25%] Building CXX object CMakeFiles/caffe2_detectron_custom_ops.dir/detectron/ops/zero_even_op.cc.o
[ 37%] Linking CXX shared library libcaffe2_detectron_custom_ops.so
/bin/ld: /mnt/lustre/zhushiwei/anaconda3/envs/DensePose/lib/libprotobuf.a(common.o): unrecognized relocation (0x2a) in section `.text'
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [libcaffe2_detectron_custom_ops.so] Error 1
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
make: *** [all] Error 2

i tried reinstall protobuf3.6.1, but it didn't work.
look forward to your reply.

Best Wishes!

I have passed the caffe2 installation test as well as the $DENSEPOSE/detectron/tests/test_spatial_narrow_as_op.py test. However for testing $DESNEPOSE/detectron/tests/test_zero_even_op.py, I get the following error:

Kindly help me resolve this issue. Thanks in advance :)

Main error message:
/usr/bin/ld: cannot find -lcaffe2_library
/usr/bin/ld: cannot find -lcaffe2_gpu_library

(caffe2) kevin@ubuntu:~/project/DensePose$ make ops
mkdir -p build && cd build && cmake .. && make -j8
-- Caffe2: CUDA detected: 10.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda-10.1/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-10.1
-- Caffe2: Header version is: 10.1
-- Found cuDNN: v7.6.5 (include: /usr/local/cuda-10.1/include, library: /usr/local/cuda-10.1/lib64/libcudnn.so)
-- Autodetected CUDA architecture(s): 7.5 7.5
-- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75
-- Summary:
-- CMake version : 3.5.1
-- CMake command : /usr/bin/cmake
-- System name : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 5.4.0
-- CXX flags : -std=c++14 -O2 -fPIC -Wno-narrowing
-- Caffe2 version : 1.4.0
-- Caffe2 include path : /home/kevin/anaconda3/envs/caffe2/lib/python2.7/site-packages/torch/include
-- Caffe2 found CUDA : True
-- CUDA version : 10.1
-- CuDNN version : 7.6.5
-- Configuring done
-- Generating done
-- Build files have been written to: /home/kevin/project/DensePose/build
make[1]: Entering directory '/home/kevin/project/DensePose/build'
make[2]: Entering directory '/home/kevin/project/DensePose/build'
make[3]: Entering directory '/home/kevin/project/DensePose/build'
make[3]: Leaving directory '/home/kevin/project/DensePose/build'
make[3]: Entering directory '/home/kevin/project/DensePose/build'
make[3]: Entering directory '/home/kevin/project/DensePose/build'
[ 12%] Linking CXX shared library libcaffe2_detectron_custom_ops.so
make[3]: Leaving directory '/home/kevin/project/DensePose/build'
make[3]: Entering directory '/home/kevin/project/DensePose/build'
/usr/bin/ld: cannot find -lcaffe2_library
[ 25%] Linking CXX shared library libcaffe2_detectron_custom_ops_gpu.so
collect2: error: ld returned 1 exit status
CMakeFiles/caffe2_detectron_custom_ops.dir/build.make:120: recipe for target 'libcaffe2_detectron_custom_ops.so' failed
make[3]: *** [libcaffe2_detectron_custom_ops.so] Error 1
make[3]: Leaving directory '/home/kevin/project/DensePose/build'
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops.dir/all' failed
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
/usr/bin/ld: cannot find -lcaffe2_gpu_library
collect2: error: ld returned 1 exit status
CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/build.make:1335: recipe for target 'libcaffe2_detectron_custom_ops_gpu.so' failed
make[3]: *** [libcaffe2_detectron_custom_ops_gpu.so] Error 1
make[3]: Leaving directory '/home/kevin/project/DensePose/build'
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all' failed
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all] Error 2
make[2]: Leaving directory '/home/kevin/project/DensePose/build'
Makefile:127: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/kevin/project/DensePose/build'
Makefile:13: recipe for target 'ops' failed
make: *** [ops] Error 2

Hello.

With your help I successfully installed densepose for the first time. Thank you very much for your information.

However, I have noticed that the installation would be easier for the latest version of pytorch binaries (installed via conda, pytorch-1.1.0 with python 3.7). The new binaries are built with protobuf 3.6.1, which can be installed through conda. Therefore it is no longer necessary to install protobuf 3.5.0 manually.

Additionally, I have noticed that some paths you provided are different from the path I figured out. I list them here just in case.
I checked the version of protobuf in $TORCH_PATH/include/caffe2/proto rather than $TORCH_PATH/lib/include/caffe2/proto . Similarly, I copied the source code directories of PyTorch to $TORCH_PATH/include/caffe2/utils/ instead of $TORCH_PATH/lib/include/caffe2/utils/. (not within the 'lib' directory)

I hope this would be helpful for the new-comers. Thank you once again!

cmake: /home/rishilss/Downloads/gcc-4.9.2/build/lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by cmake) cmake: /home/rishilss/Downloads/gcc-4.9.2/build/lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by cmake)
cmake: /home/rishilss/Downloads/gcc-4.9.2/build/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /usr/lib/x86_64-linux-gnu/libjsoncpp.so.1)

I followed the exact steps mentioned in the guide but ran into this error when compiling the custom operator. Please help.

The test shows the following results
net_drawer will not run correctly. Please install the correct dependencies. [E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. [E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. [E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. Found Detectron ops lib: /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/torch/lib/libcaffe2_detectron_ops_gpu.so terminate called after throwing an instance of 'c10::Error' what(): [enforce fail at context_gpu.h:206] error == cudaSuccess. 2 vs 0. Error at: /opt/conda/conda-bld/pytorch_1556653114079/work/caffe2/core/context_gpu.h:206: out of memory frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x59 (0x7f97ef1fd409 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/../../torch/lib/libc10.so) frame #1: <unknown function> + 0xe5456 (0x7f97ef9b5456 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #2: <unknown function> + 0xe5648 (0x7f97ef9b5648 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #3: <unknown function> + 0xe86b2 (0x7f97ef9b86b2 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #4: <unknown function> + 0xea608 (0x7f97ef9ba608 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #5: <unknown function> + 0x4ee2c (0x7f97ef91ee2c in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #6: <unknown function> + 0x8a6ff (0x7f97ef95a6ff in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) frame #7: <unknown function> + 0x8c928 (0x7f97ef95c928 in /data1/liuguanze/anaconda3/envs/densepose/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-37m-x86_64-linux-gnu.so) <omitting python frames>
I'm now running densepose with python3 support as you suggested.
my dependencies are:
python 3.7
pytorch 1.1.0
gcc 4.9.2
protobuf 3.6.1
cmake 3.8.2
Is there any insights about what happended?

There seems to be a problem with make ops when running on pytorch 1.2. The issue appears to be similar to issue 238 in the densepose original repository.

Is it possible if you can do a video tutorial?

python tools/infer_simple.py     --cfg configs/DensePose_ResNet101_FPN_s1x-e2e.yaml     --output-dir DensePoseData/infer_out/     --image-ext jpg     --wts https://dl.fbaipublicfiles.com/densepose/DensePose_ResNet101_FPN_s1x-e2e.pkl     DensePoseData/demo_data/demo_im.jpg
/home/SENSETIME/zhushiwei/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Found Detectron ops lib: /home/SENSETIME/zhushiwei/anaconda3/lib/python3.6/site-packages/torch/lib/libcaffe2_detectron_ops_gpu.so
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
  File "tools/infer_simple.py", line 176, in <module>
    main(args)
  File "tools/infer_simple.py", line 117, in main
    args.weights = cache_url(args.weights, cfg.DOWNLOAD_CACHE)
  File "/home/SENSETIME/zhushiwei/sense_work/DensePose/detectron/utils/io.py", line 100, in cache_url
    'bucket: {}').format(_DETECTRON_S3_BASE_URL)
AssertionError: Detectron only automatically caches URLs in the Detectron S3 bucket: https://dl.fbaipublicfiles.com/detectron

i download git clone https://github.com/Johnqczhang/DensePose $DENSEPOSE
run the code than i recieve the following code
how can i fix this problem?
thx!!!!