Build failure when using CUDAGraph about pytorch

[malfet]

This sounds like a documentation issue. In my opinion, libtorch should not be redistributed with CUDA headers, is they are not really needed if one want to use PyTorch with GPU acceleration from their application. But if one wants to build CUDA extensions or use some advanced features, then they must install coda-toolkit and modify CMakeFile to specify path to cuda headers/libraries. Adding module: docs to perhaps extend/reference cppdocs example to an example that can use both PyTorch and CUDA

cc: @msaroufim

[nWEIdia]

What is the output of running the following?

cmake -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'` ..

[nWEIdia]

/usr/local/cuda/include should include this file. Perhaps, cuda needs to be installed first.

[aseyboldt]

For me the file is in /opt/cuda/include/cuda_runtime_api.h, and I could for instance add this to the include path to make this work for me locally.
But nothing else (as far as I know) in pytorch assumes that cuda headers are available at compile time, so I can usually just write code involving cuda, and that code will compile fine even if cuda is not available.

For instance the following compiles without issues (throws an error at runtime of course)

#include <torch/torch.h>
#include <iostream>

int main() {
  torch::Device device(torch::kCUDA);
  torch::Tensor tensor = torch::rand({2, 3}).to(device);
  std::cout << tensor << std::endl;
}

This is quite nice when distributing code, because the question if cuda is available or not can be dealt with at runtime. This suddenly changes when I include the CUDAGraph header however.

Looks to me like this include here is the cause, and this might for the most part be an oversight that this ended up in the public header?

I just noticed that this issue looks like a duplicate of this by the way: #55454 (sorry for not checking more carefully)
I think I'm running into this problem in very similar circumstances.

[nWEIdia]

Thanks @aseyboldt for the nice explanation and root-cause. I would like to defer this to @cyyever
It is possible that the new change might need some update on the builder or packaging side.

Also cc @ptrblck @atalman @malfet

[aseyboldt]

Hm, but shouldn't that then at least be checked in cmake? For instance, the cuda versions of libtorch and the user provided cuda header still need to match, don't they?

I have to admit I completely missed, that it seems I can use c10/core/Stream.h, which I think should work without cuda headers? Unless I'm missing something that should fix my issues. (And maybe that also works for tch-rs? cc @LaurentMazare, if that is still an issue for you...)