yuejiang-nj / cvpr2020-sdfdiff Goto Github PK

Hello, I am trying to follow the readme to run the example code provided. When i run setup.py in both multiview and singleview, i run into this error:

C:\ProgramData\Anaconda3\envs\sdfdiff\lib\site-packages\torch\utils\cpp_extension.py:346: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
error: [WinError 2] The system cannot find the file specified

any ideas what might be causing this? am I missing a package or something?

thanks

Watching some of those 3D ML historical photo reconstructions on YouTube made me wonder if we could turn Muybridge's "The Horse in Motion" into an SDF model:

I guess there are numerous technical reasons why not, but just an idea.

Hi @YueJiang-nj , thank you for sharing the code of your nice work.
I have a question about your approximation of an intersection point.

In my understanding, the core idea of your paper is that for computing an intersection point, constructing the computation graph through all the ray marching iterations is not needed .
So, you approximate the intersection point using trilinear interpolation between the eight local samples.

Is a simple approximation of an intersection point like p=s+SDF(s)v problematic?
Why did you use the eight local samples some of which are not on the ray?

The notation follows your paper.

Thank you.

Hi,
Thank you for releasing the codes for your great job in the differentiable SDF rendering.

Currently, I am testing the codes following the instructions. I am confused with some issues as:

1. At line. 8 in the 'main.py' in the 'single_view_code' directory, the code indicates to import the 'dataset' module, but there is no correspoinding file named 'dataset' in the project.
2. At line. 7 in the 'main.py' in the 'single_view_code' directory, the code tries to import 'Encoder' and 'Decoder' from 'models.py'. However, the corresponding classes are missing in 'models.py'. Instead, I found the classes 'DecoderSimple' and 'EncoderSimple'. Do 'Encoder' and 'Decoder' mean 'DecoderSimple' and 'EncoderSimple'?

Hi, Jiang Yue, Congratulation on your work! It is a clever idea to combine rendering with optimizing SDF.

• A tiny error

In multi_view_code/code/main.py 754 line:
grid_res_update_x = grid_res_update_y = grid_res_update_z = voxel_res_list.pop(0)
Eventually voxel_res_list will pop from an empty list and raise error.

• I wonder what if light move across the surface . To my understanding, SDF is not accurate during optimizing, so sphere marching might not guarantee that light never move across the boundary of the object.

• The direct result of multi-view optimization seems a bit noisy (in rendering image), do you reconstruct the mesh from the SDF field? Please tell me if I missed something. Thanks!

Hello.

I am not able to complete the build process by using the setup.py file. I am getting the following error:

May I know how to resolve this error?

Hi @YueJiang-nj , congratulation on your great work! I was reading your paper and there are two questions I didn't understand well. It would be great if you can let me know a bit more details about them:

1. As you mentioned in the paper, the selection of the 8 grid points is based on ray intersection point. So if there is no intersection, no gradient will be backpropagated. Then at the beginning, I'm assuming there will be cases where the groundtruth shape does not intersect with the initial shape. In such a case, how can you prevent the model from failing to learn?

2. It seems that you are using Lambertian shading to get the final image from SDF. For single-image shape reconstruction, do you also directly use groundtruth lighting as input? And textures are not considered in this paper right?

Thank you!

Hi! Very interesing paper!
But I can't reproduce the code.
It does not compile. (Ubuntu 20, CUDA 10.2, pytorch=1.5, python=3.7)

Could you please outline the exact prerequisites like in
https://github.com/B1ueber2y/DIST-Renderer

Just in case, maybe you could see what's wrong here, here is output of compilation:

`
running install
running bdist_egg
running egg_info
writing renderer.egg-info/PKG-INFO
writing dependency_links to renderer.egg-info/dependency_links.txt
writing top-level names to renderer.egg-info/top_level.txt
reading manifest file 'renderer.egg-info/SOURCES.txt'
writing manifest file 'renderer.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'renderer' extension
Emitting ninja build file /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] /usr/bin/nvcc -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/home/ab/anaconda3/include/python3.7m -c -c /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer_kernel.cu -o /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/renderer_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=renderer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
FAILED: /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/renderer_kernel.o
/usr/bin/nvcc -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/home/ab/anaconda3/include/python3.7m -c -c /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer_kernel.cu -o /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/renderer_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=renderer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
/usr/include/c++/8/utility(307): error: pack expansion does not make use of any argument packs

/usr/include/c++/8/utility(329): error: pack expansion does not make use of any argument packs

/usr/include/c++/8/utility(329): error: expected a ">"
detected during instantiation of type "std::make_integer_sequence<std::size_t, _Num>"
(340): here

/usr/include/c++/8/utility(329): error: identifier "__integer_pack" is undefined
detected during:
instantiation of type "std::make_integer_sequence<std::size_t, 0UL>"
(340): here
instantiation of type "std::make_index_sequence<0UL>"
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/op_registration/infer_schema.h(81): here
instantiation of "std::array<c10::detail::infer_schema::ArgumentDef, > c10::detail::infer_schema::createReturns<std::tuple<ReturnTypes...>, void>::call() [with ReturnTypes=<>]"
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/op_registration/infer_schema.h(96): here

/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/op_registration/infer_schema.h(80): error: no instance of function template "c10::detail::infer_schema::createArgumentVectorFromTypes" matches the argument list
argument types are: ()
detected during instantiation of "std::array<c10::detail::infer_schema::ArgumentDef, > c10::detail::infer_schema::createReturns<std::tuple<ReturnTypes...>, void>::call() [with ReturnTypes=<>]"
(96): here

5 errors detected in the compilation of "/tmp/tmpxft_00008345_00000000-6_renderer_kernel.cpp1.ii".
[2/2] c++ -MMD -MF /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/renderer.o.d -pthread -B /home/ab/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/home/ab/anaconda3/include/python3.7m -c -c /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp -o /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/build/temp.linux-x86_64-3.7/renderer.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=renderer -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp: In function ‘std::vectorat::Tensor ray_matching(at::Tensor, at::Tensor, at::Tensor, int, int, float, float, float, float, float, float, int, int, int, float, float, float)’:
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:41: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:164:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
#define C10_EXPAND_MSVC_WORKAROUND(x) x
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:183:34: note: in expansion of macro ‘C10_UNLIKELY’
#define C10_UNLIKELY_OR_CONST(e) C10_UNLIKELY(e)
^~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:220:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
if (C10_UNLIKELY_OR_CONST(!(cond))) {
^~~~~~~~~~~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:410:32: note: in expansion of macro ‘TORCH_INTERNAL_ASSERT’
C10_EXPAND_MSVC_WORKAROUND(TORCH_INTERNAL_ASSERT(cond, VA_ARGS));
^~~~~~~~~~~~~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:23: note: in expansion of macro ‘AT_ASSERTM’
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:25:24: note: in expansion of macro ‘CHECK_CUDA’
#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:45:5: note: in expansion of macro ‘CHECK_INPUT’
CHECK_INPUT(w_h_3);
^
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:11:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:5,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:262:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:41: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:164:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
#define C10_EXPAND_MSVC_WORKAROUND(x) x
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:183:34: note: in expansion of macro ‘C10_UNLIKELY’
#define C10_UNLIKELY_OR_CONST(e) C10_UNLIKELY(e)
^~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:220:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
if (C10_UNLIKELY_OR_CONST(!(cond))) {
^~~~~~~~~~~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:410:32: note: in expansion of macro ‘TORCH_INTERNAL_ASSERT’
C10_EXPAND_MSVC_WORKAROUND(TORCH_INTERNAL_ASSERT(cond, VA_ARGS));
^~~~~~~~~~~~~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:23: note: in expansion of macro ‘AT_ASSERTM’
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:25:24: note: in expansion of macro ‘CHECK_CUDA’
#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:46:5: note: in expansion of macro ‘CHECK_INPUT’
CHECK_INPUT(w_h);
^
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:11:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:5,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:262:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:41: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:164:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
#define C10_EXPAND_MSVC_WORKAROUND(x) x
^
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:183:34: note: in expansion of macro ‘C10_UNLIKELY’
#define C10_UNLIKELY_OR_CONST(e) C10_UNLIKELY(e)
^~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:220:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
if (C10_UNLIKELY_OR_CONST(!(cond))) {
^~~~~~~~~~~~~~~~~~~~~
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:410:32: note: in expansion of macro ‘TORCH_INTERNAL_ASSERT’
C10_EXPAND_MSVC_WORKAROUND(TORCH_INTERNAL_ASSERT(cond, VA_ARGS));
^~~~~~~~~~~~~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:23:23: note: in expansion of macro ‘AT_ASSERTM’
#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:25:24: note: in expansion of macro ‘CHECK_CUDA’
#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
^~~~~~~~~~
/home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:47:5: note: in expansion of macro ‘CHECK_INPUT’
CHECK_INPUT(grid);
^
In file included from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:11:0,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:5,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
from /home/ab/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/ab/Documents/CVPR2020-SDFDiff/single_view_code/renderer.cpp:1:
/home/ab/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:262:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/ab/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1400, in _run_ninja_build
check=True)
File "/home/ab/anaconda3/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "setup.py", line 13, in
'build_ext': BuildExtension
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/home/ab/anaconda3/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/ab/anaconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/ab/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/home/ab/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/ab/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 172, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 158, in call_command
self.run_command(cmdname)
File "/home/ab/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/ab/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/ab/anaconda3/lib/python3.7/distutils/command/install_lib.py", line 107, in build
self.run_command('build_ext')
File "/home/ab/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/ab/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
_build_ext.run(self)
File "/home/ab/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/ab/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/ab/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
build_ext.build_extensions(self)
File "/home/ab/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/ab/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/home/ab/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/ab/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
_build_ext.build_extension(self, ext)
File "/home/ab/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/home/ab/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 423, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/ab/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/ab/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
raise RuntimeError(message)
RuntimeError: Error compiling objects for extension
`

Hi, thanks for your paper and code. I have some questions about the claim in the paper:

A key observation is that the derivatives of a given pixel with respect to rendering parameters only depend on a local neighborhood of eight SDF samples that define the value of the trilinearly interpolated SDF at the surface intersection point.

I'm a little bit confused about it. Suppose we do sphere tracing twice to get the ray-surface intersection point q from the start point p, then we have q = p1 + Trilinear(d0,...,d7 ; p1) where p1 = p + Trilinear(d0',...,d7'; p). So the ray-surface intersection point q should have a dependency not only on its local neighborhood (i.e. d0,...,d7), but also on the neighborhood along the sphere tracing path (in this case should be d0',...,d7').

In other words, the shading loss is dependent on the normal of the ray-surface intersection point q; the normal is dependent on the XYZ position of q; the position of q is dependent on all the neighborhood grid along the sphere tracing path.
Is it means that the SDFDiff method only consider d(q)/d(d0),..., d(q)/d(d7), should we also need to consider d(q)/d(d0'),..., d(q)/d(d7')?

Thanks for sharing the source code!

I have a question regarding how the grid_normal is computed at here. To compute the grid_normal for a grid vertex with central finite difference, I was expecting that the 6 adjacent vertices will be used. However, the code seems to be doing something different? Could you please explain a bit more?

In addition, if the grid_normal of a grid vertex involves 6 adjacent vertices, then the normal at intersection point involves 8x6-8x2=32 vertices? 8x6 is because the normal at intersection point is acquired by trilinearly interpolating the grid_normal of its 8 closest vertices; subtracting 8x2 is because each of these 8 closest vertices is counted three times. However, the paper says 4x4x4=64 vertices are involved. Am I missing something? Could you please elaborate?

Thank you!

Can this be used on photographs, given camera positions, to generate mesh output?

Hi,

I wanted to try your code, however, the dataset.py file appears to be missing.
Could you provide this file?
Also, is there a test script?

Best
Felix Petersen

Thanks for sharing the source code. I noticed this line in your code. I just want to double check that in your multi-view code, the directional light for each training image is aligned with the camera, hence varies across different training views, right? In other words, the lighting is not fixed for the 26 training images?

Thank you!

Thank you very much for publishing your code. Very interesting work.

I am interested in using the rendering pipeline for batched computation. More exactly, suppose I have a tensor of size B x NxNxN
which represents a mini batch of data containing B SDF grids of size N^3. My understanding is that the current implementation does not support batch computation (and I would need to do the computation in a loop). I was wondering if my understanding is correct, and if so, if there is any easy way to modify the implementation to support this feature.

Thanks,

hello, Is there ppt that explain the paper, I find it is hard for me to understand your paper.