CUDA error: CUBLAS_STATUS_EXECUTION_FAILED about deepsvg

[alexandre01]

Hi @pwichmann ,

I'm doing a fresh install right now to check if I get the same issue. Otherwise, will try to investigate the error message you are receiving.

[alexandre01]

Okay, installed everything with the following configuration:

• Ubuntu 20.04
• CUDA 11.0
• GPU GTX 1080

and worked without error... Trying to check your error message now

[alexandre01]

Hmm.. Yeah, googling didn't help me much neither. I suspect this might be a problem with PyTorch 1.4 not supporting your exact CUDA build. It's tricky to give any advice here cause I always just install CUDA using Ubuntu's Software & Updates (additional drivers) tool. I don't have any RTX card neither..

[pwichmann]

I was able to solve the problem. It was NOT caused by your code but - indeed - by the PyTorch version or CUDA version.
A fresh install of PyTorch with the most recent version solved the problem.

pip install torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

[pwichmann]

The same problem occurs in notebooks/latent_ops.ipynb in the cell "Addition" operation.

z_list = []

for i in range(500):
    tensors, fillings = dataset._load_tensor(dataset.random_id())
    t_sep = tensors[0]
    t_sep_rm, fillings_rm = t_sep[:-1], fillings[:-1]

    if len(t_sep) >= 2:
        z1 = encode(dataset.get_data(t_sep, fillings))
        z2 = encode(dataset.get_data(t_sep_rm, fillings_rm))
        z_list.append(z2 - z1)
z_rmv = torch.cat(z_list).mean(dim=0, keepdims=True)

[pwichmann]

Thanks, @alexandre01

One difference is the CUDA version. But I could not determine if this can be the cause.

[pwichmann]

Thank you, @alexandre01!

[alexandre01]

Wow great, I'm glad you managed to have it working!