Many thanks for this amazing repo!

Unfortunately, I was unable to build diffvg.
I am not an expert (in anything) -- are you able to understand what went wrong from the error message below?

Environment

• Ubuntu 20.04
• Python 3.9.2
• Conda environment

Error log

python setup.py build --debug install
running build
running build_py
running build_ext
CMake Error at CMakeLists.txt:13 (add_subdirectory):
  The source directory

    /home/user/Documents/01_quiv/09_diffvg/diffvg/pybind11

  does not contain a CMakeLists.txt file.


-- Build with CUDA support
INFO Building without TensorFlow support (not found)
-- Configuring incomplete, errors occurred!
See also "/home/user/Documents/01_quiv/09_diffvg/diffvg/build/temp.linux-x86_64-3.9/CMakeFiles/CMakeOutput.log".
See also "/home/user/Documents/01_quiv/09_diffvg/diffvg/build/temp.linux-x86_64-3.9/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
  File "/home/user/Documents/01_quiv/09_diffvg/diffvg/setup.py", line 91, in <module>
    setup(name = 'diffvg',
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/site-packages/setuptools/__init__.py", line 163, in setup
    return distutils.core.setup(**attrs)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/user/Documents/01_quiv/09_diffvg/diffvg/setup.py", line 31, in run
    super().run()
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/user/Documents/01_quiv/09_diffvg/diffvg/setup.py", line 64, in build_extension
    subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
  File "/home/user/anaconda3/envs/im2vec/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/home/user/Documents/01_quiv/09_diffvg/diffvg', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user/Documents/01_quiv/09_diffvg/diffvg/build/lib.linux-x86_64-3.9', '-DPYTHON_INCLUDE_PATH=/home/user/anaconda3/envs/im2vec/include/python3.9', '-DCMAKE_BUILD_TYPE=Debug', '-DDIFFVG_CUDA=1']' returned non-zero exit status 1.

Greetings! I would like to know, if background_image is passed to render, and background_image itself is a tensor in the computation graph that requires gradient, if the gradient flow is maintained through the background_image.

Thanks!

Hi, I am trying the sketch GAN example, but failed to run it because of RuntimeError: The total length of the shape boundaries in the scene is not a number. Length = -nan.

If I run the following code for example:

g = Generator()
g(torch.randn([1, 100]))

I got:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-7f54cfbd38f5> in <module>
      1 g = Generator()
      2 t = torch.randn([1, 100])
----> 3 g(t)

~/miniconda3/envs/diffvg/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

<ipython-input-1-3543db5b47a9> in forward(self, z)
     93                          random.randint(0, 1048576), # seed
     94                          None,
---> 95                          *scene_args)
     96             img = img[:, :, :1]
     97             # HWC -> NCHW

~/miniconda3/envs/diffvg/lib/python3.7/site-packages/pydiffvg/render_pytorch.py in forward(ctx, width, height, num_samples_x, num_samples_y, seed, background_image, *args)
    366         scene = diffvg.Scene(canvas_width, canvas_height,
    367             shapes, shape_groups, filt, pydiffvg.get_use_gpu(),
--> 368             pydiffvg.get_device().index if pydiffvg.get_device().index is not None else -1)
    369         time_elapsed = time.time() - start
    370         global print_timing

RuntimeError: The total length of the shape boundaries in the scene is not a number. Length = -nan

Any idea why?

Recently I decided to play with Word-As-Image, which depends on diffvg project. I pulled the repo and tried to follow the python setup.py install instruction, but encounters a few problems. I list them below, for developers to consult with.

1. NoneType error as in #57. This error comes from the undefined LIBDIR config variable. I'm using a Miniconda3 / python 3.8 environment, and not sure if the absence of it is abnormal, but I found another variable called LIBDEST, and replaced the former one.
2. cmake fail with

subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '/m']' returned non-zero exit status 1.

I located the error in subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp) in setup.py, but first assumed that this is caused by not using CUDA. I rewrote some lines in setup.py, in vain. Then I assumed some cmake config error and looked at the cmake prompts, no use. The final solution lies in the error message

(path\to)\diffvg\diffvg.h(97,15): error C2169: 'log2': intrinsic function, cannot be defined [(path\to)\diffvg\build\temp.win-amd64-cpython-38\Release\diffvg.vcxproj]

and I commented out the inline double log2(double x) { .. } function in diffvg.h. It works.
3. diffvg import error. I found some reference to it here, and solved it with #12 (comment) .

Hello! I am trying to get fully-reproducible pipeline, however with all seeds fixed i got different grad values on first iteration in single_circle.py app even with same loss value:

Fixing all seeds:

import random
import os

seed = 0
torch.set_printoptions(precision=16)
os.environ['PYTHONHASHSEED'] = str(seed)
torch.manual_seed(seed)
random.seed(seed)
np.random.seed(seed)
torch.backends.cudnn.benchmark = False
torch.use_deterministic_algorithms(True)
os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.enabled = False
torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
torch.set_deterministic_debug_mode(2)

Run1 python single_circle.py

iteration: 0
loss: 6360.21044921875
radius.grad: tensor(-6679.3833007812500000)
center.grad: tensor([-16965.3789062500000000,   8691.6396484375000000])

Run2 python single_circle.py

iteration: 0
loss: 6360.21044921875
radius.grad: tensor(-6679.9208984375000000)
center.grad: tensor([-16965.9726562500000000,   8691.4550781250000000])

When I run refine_svg.py on an svg file created with painterly_rendering.py I get the following error:

LPIPS is untested <pydiffvg.shape.ShapeGroup object at 0x000001CFC58F1BE0> Traceback (most recent call last): File "refine_svg.py", line 116, in <module> main(args) File "refine_svg.py", line 41, in main group.fill_color.requires_grad = True AttributeError: 'NoneType' object has no attribute 'requires_grad'

Commands that I used:
python painterly_rendering.py mypicture.jpg --num_paths 2048 --max_width 4.0 --use_lpips_loss
python refine_svg.py --use_lpips_loss mypicture.svg mypicture.jpg

Environment: Windows + Conda
Any suggestions are appreciated.

EDIT:
I tried again by creating the svg with painterly_rendering.py and using the --use_blob
After doing that I was able to run refine_svg.py on the .svg
I still don't know what the problem was however.

Does anyone encounter the issue when rendering the GPU?

For my case, when executing the app/single_curve.py with GPU, it will render blank output.

• cuda: 11.7 ( also I use coda-toolkit with 11.7)
• ubuntu 22.04

Python version 3.7, Pytorch with Cuda (confirmed working), VS 2019 CMake installed. Cuda 10.0 - 10.2 installed from Nvidia site. I cannot compile diffvg library with Cuda enabled. I get this error every time:
ptxas fatal : Unresolved extern function '_Z3powfi'
diffvg.cpp
CMake Error at diffvg_generated_diffvg.cpp.obj.Release.cmake:280 (message):
Error generating file
......./build/temp.win-amd64-3.6/Release/CMakeFiles/diffvg.dir//Release/diffvg_generated_diffvg.cpp.obj

Edit: It looks like I need to make a few changes to the way drivers are referenced. I was able to get this working under linux for now.

Thanks for your outstanding contribution！

I encountered some errors while installing@BachiLi.

Could you please help me？

Here's the information.Thank you so much!

Building NVCC (Device) object CMakeFiles/diffvg.dir/Release/diffvg_generated_diffvg.cpp.obj
diffvg.cpp
C:\Users\LENOVO\Desktop\diffvg-master\pybind11\include\pybind11\cast.h(1405): error : too few arguments for template te
mplate parameter "Tuple" [C:\Users\LENOVO\Desktop\diffvg-master\build\temp.win-amd64-3.8\Release\diffvg.vcxproj]
detected during instantiation of class "pybind11::detail::tuple_caster<Tuple, Ts...> [with Tuple=std::pair,
Ts=<T1, T2>]"
(1483): here

C:\Users\LENOVO\Desktop\diffvg-master\pybind11\include\pybind11\cast.h(1479): error : too few arguments for template te
mplate parameter "Tuple" [C:\Users\LENOVO\Desktop\diffvg-master\build\temp.win-amd64-3.8\Release\diffvg.vcxproj]
detected during instantiation of class "pybind11::detail::tuple_caster<Tuple, Ts...> [with Tuple=std::pair,
Ts=<T1, T2>]"
(1483): here

C:\Users\LENOVO\Desktop\diffvg-master\pybind11\include\pybind11\cast.h(1041): warning : pointless comparison of unsigne
d integer with zero [C:\Users\LENOVO\Desktop\diffvg-master\build\temp.win-amd64-3.8\Release\diffvg.vcxproj]
detected during:
instantiation of "nv_bool pybind11::detail::type_caster<T, std::enable_if_t<, void>>::load(
pybind11::handle, nv_bool) [with T=pybind11::detail::intrinsic_t<size_t>]"
(2004): here
instantiation of "nv_bool pybind11::detail::argument_loader<Args...>::load_impl_sequence(pybind11::deta
il::function_call &, std::index_sequence<Is...>) [with Args=<pybind11::detail::value_and_holder &, size_t>, Is=<0ULL,
1ULL>]"
(1980): here
instantiation of "nv_bool pybind11::detail::argument_loader<Args...>::load_args(pybind11::detail::funct
ion_call &) [with Args=<pybind11::detail::value_and_holder &, size_t>]"
C:/Users/LENOVO/Desktop/diffvg-master/pybind11/include\pybind11/pybind11.h(159): here
instantiation of "void pybind11::cpp_function::initialize(Func &&, Return (*)(Args...), const Extra &...)
[with Func=lambda [](pybind11::detail::value_and_holder &, size_t)->void, Return=void, Args=<pybind11::detail::value
and_holder &, size_t>, Extra=<pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style
constructor>]"
C:/Users/LENOVO/Desktop/diffvg-master/pybind11/include\pybind11/pybind11.h(72): here
instantiation of "pybind11::cpp_function::cpp_function(Func &&, const Extra &...) [with Func=lambda [](py
bind11::detail::value_and_holder &, size_t)->void, Extra=<pybind11::name, pybind11::is_method, pybind11::sibling, pyb
ind11::detail::is_new_style_constructor>, =void]"
C:/Users/LENOVO/Desktop/diffvg-master/pybind11/include\pybind11/pybind11.h(1162): here
instantiation of "pybind11::class<type, options...> &pybind11::class<type, options...>::def(const cha
r *, Func &&, const Extra &...) [with type=ptr, options=<>, Func=lambda [](pybind11::detail::value_and_holder
&, size_t)->void, Extra=pybind11::detail::is_new_style_constructor]"
C:\Users\LENOVO\Desktop\diffvg-master\pybind11\include\pybind11\detail/init.h(176): here
instantiation of "void pybind11::detail::initimpl::constructor<Args...>::execute(Class &, const Extra &..
.) [with Args=<size_t>, Class=pybind11::class<ptr>, Extra=<>, =0]"
C:/Users/LENOVO/Desktop/diffvg-master/pybind11/include\pybind11/pybind11.h(1191): here
instantiation of "pybind11::class<type, options...> &pybind11::class<type, options...>::def(const pyb
ind11::detail::initimpl::constructor<Args...> &, const Extra &...) [with type_=ptr, options=<>, Args=<size_t>,
Extra=<>]"
c:\users\lenovo\desktop\diffvg-master\diffvg.cpp(1655): here

2 errors detected in the compilation of "c:/users/lenovo/desktop/diffvg-master/diffvg.cpp".
diffvg.cpp
CMake Error at diffvg_generated_diffvg.cpp.obj.Release.cmake:280 (message):
Error generating file
C:/Users/LENOVO/Desktop/diffvg-master/build/temp.win-amd64-3.8/Release/CMakeFiles/diffvg.dir//Release/diffvg_genera
ted_diffvg.cpp.obj

Traceback (most recent call last):
File "setup.py", line 91, in
setup(name = 'diffvg',
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools_init_.py", line 153, in setup
return distutils.core.setup(**attrs)
File "E:\miniconda\envs\diffvg\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "E:\miniconda\envs\diffvg\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "E:\miniconda\envs\diffvg\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\install.py", line 67, in run
self.do_egg_install()
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "E:\miniconda\envs\diffvg\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "E:\miniconda\envs\diffvg\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "E:\miniconda\envs\diffvg\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "E:\miniconda\envs\diffvg\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\install_lib.py", line 11, in run
self.build()
File "E:\miniconda\envs\diffvg\lib\distutils\command\install_lib.py", line 107, in build
self.run_command('build_ext')
File "E:\miniconda\envs\diffvg\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "E:\miniconda\envs\diffvg\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 31, in run
super().run()
File "E:\miniconda\envs\diffvg\lib\site-packages\setuptools\command\build_ext.py", line 79, in run
_build_ext.run(self)
File "E:\miniconda\envs\diffvg\lib\distutils\command\build_ext.py", line 340, in run
self.build_extensions()
File "E:\miniconda\envs\diffvg\lib\distutils\command\build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "E:\miniconda\envs\diffvg\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "setup.py", line 65, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
File "E:\miniconda\envs\diffvg\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '/m']' returned non-zero exit status 1.

Thank you for publishing the source. Could you please add a LICENSE file?
I'm making a package for Arch Linux.

Hi,
I was trying to modify the single_circle_outline.py to optimize a ring shape by setting the fill color of a circle to None and using a nonzero stroke width. But the optimization never converges, and the radius of the circle always shrinks to a value smaller than 0. So I rendered a gradient map for pixel color wrt the radius parameter, and realize its value disagrees with finite-difference. Specifically, for the inner circle, its gradient has an opposite sign as in FD. I'm guessing it's a bug in the implementation, but also want to check whether this behavior is caused by the math derivation?
Code that generates disagreeing gradient map for diffvg and finite diff:

import pydiffvg
import diffvg
import torch
import skimage
import numpy as np
import os
import time
import platform
import sys

# Use GPU if available
pydiffvg.set_use_gpu(torch.cuda.is_available())
render = pydiffvg.RenderFunction.apply

def ring():
    circle = pydiffvg.Circle(radius = torch.tensor(80.0),
                             center = torch.tensor([128.0, 128.0]),
                             stroke_width = torch.tensor(16.0))
    
    shapes = [circle]
    circle_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor([0]),
        fill_color = None,
        stroke_color = torch.tensor([1.0, 0.3, 0.6, 1.0]))
    shape_groups = [circle_group]
    
    return shapes, shape_groups

def get_img(shapes, shape_groups, width, height, radius_val, nsamples=2, seed=None):

    shapes[0].radius = radius_val
    
    scene_args = pydiffvg.RenderFunction.serialize_scene(\
        width, height, shapes, shape_groups)
    
    img = render(width, # width
                 height, # height
                 nsamples,   # num_samples_x
                 nsamples,   # num_samples_y
                 seed if seed is not None else 0,   # seed
                 None,
                 *scene_args)
    
    return img

def main():
    
    if '--res_x' in sys.argv:
        res_x_idx = sys.argv.index('--res_x')
        res_x = int(sys.argv[res_x_idx + 1])
    else:
        res_x = 256
        
    if '--res_y' in sys.argv:
        res_y_idx = sys.argv.index('--res_y')
        res_y = int(sys.argv[res_y_idx + 1])
    else:
        res_y = 256
        
    if '--nsamples' in sys.argv:
        nsamples_idx = sys.argv.index('--nsamples')
        nsamples = int(sys.argv[nsamples_idx + 1])
    else:
        nsamples = 2
        
    canvas_width = res_x
    canvas_height = res_y
    
    outdir = ''
        
    save_dir = outdir

    canvas_width, canvas_height = 256, 256
    
    shapes, shape_groups = ring()
    
    radius_val = torch.tensor(80., requires_grad=True)
    
    img = get_img(shapes, shape_groups, res_x, res_y, radius_val, nsamples=nsamples).cpu().detach().numpy().copy()[..., 0]
    skimage.io.imsave(os.path.join(outdir, 'img.png'), img)
    
    radius_val.data += 1.
    img_pos = get_img(shapes, shape_groups, res_x, res_y, radius_val, nsamples=nsamples).cpu().detach().numpy().copy()[..., 0]
    radius_val.data -= 2.
    img_neg = get_img(shapes, shape_groups, res_x, res_y, radius_val, nsamples=nsamples).cpu().detach().numpy().copy()[..., 0]

    fd_wrt_radius = (img_pos - img_neg) / 2.
    
    skimage.io.imsave(os.path.join(outdir, 'fd_gradient_wrt_radius.png'), fd_wrt_radius)
    
    sparse_x, sparse_y = np.where(fd_wrt_radius != 0)
    
    # only compute gradient of R channel wrt radius
    diffvg_wrt_radius = np.zeros((res_x, res_y))

    radius_val.data += 1.
    for i in range(sparse_x.shape[0]):
        if radius_val.grad is not None:
            radius_val.grad.data.zero_()
        get_img(shapes, shape_groups, res_x, res_y, radius_val, nsamples=nsamples)[sparse_x[i], sparse_y[i], 0].backward()
        diffvg_wrt_radius[sparse_x[i], sparse_y[i]] = radius_val.grad
        
        if i % 100 == 0:
            print(i)
        
    # use skimage to avoid negative value being clipped
    
    skimage.io.imsave(os.path.join(outdir, 'diffvg_gradient_wrt_radius.png'), diffvg_wrt_radius)

if __name__ == '__main__':
    main()

Code to reproduce the non-convergent optimization (slight modification from single_circle_outline.py):

import pydiffvg
import torch
import skimage
import numpy as np

# Use GPU if available
pydiffvg.set_use_gpu(torch.cuda.is_available())

canvas_width, canvas_height = 256, 256
circle = pydiffvg.Circle(radius = torch.tensor(40.0),
                         center = torch.tensor([128.0, 128.0]),
                         stroke_width = torch.tensor(5.0))
shapes = [circle]
circle_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor([0]),
    fill_color = None,
    stroke_color = torch.tensor([0.6, 0.3, 0.6, 0.8]))
shape_groups = [circle_group]
scene_args = pydiffvg.RenderFunction.serialize_scene(\
    canvas_width, canvas_height, shapes, shape_groups)

render = pydiffvg.RenderFunction.apply
img = render(256, # width
             256, # height
             2,   # num_samples_x
             2,   # num_samples_y
             0,   # seed
             None,
             *scene_args)
# The output image is in linear RGB space. Do Gamma correction before saving the image.
pydiffvg.imwrite(img.cpu(), 'target.png', gamma=2.2)
target = img.clone()

# Move the circle to produce initial guess
# normalize radius & center for easier learning rate
radius_n = torch.tensor(20.0 / 256.0, requires_grad=True)
center_n = torch.tensor([108.0 / 256.0, 138.0 / 256.0], requires_grad=True)
stroke_color = torch.tensor([0.4, 0.7, 0.5, 0.5], requires_grad=True)
stroke_width_n = torch.tensor(10.0 / 100.0, requires_grad=True)
circle.radius = radius_n * 256
circle.center = center_n * 256
circle.stroke_width = stroke_width_n * 100
circle_group.stroke_color = stroke_color
scene_args = pydiffvg.RenderFunction.serialize_scene(\
    canvas_width, canvas_height, shapes, shape_groups)
img = render(256, # width
             256, # height
             2,   # num_samples_x
             2,   # num_samples_y
             1,   # seed
             None,
             *scene_args)
pydiffvg.imwrite(img.cpu(), 'init.png', gamma=2.2)

# Optimize for radius & center
optimizer = torch.optim.Adam([radius_n, center_n, stroke_color, stroke_width_n], lr=1e-2)
# Run 200 Adam iterations.
for t in range(200):
    print('iteration:', t)
    optimizer.zero_grad()
    # Forward pass: render the image.
    if radius_n < 0:
        # avoids rendering error
        radius_n.data = torch.tensor(np.float32(0.1))
    circle.radius = radius_n * 256
    circle.center = center_n * 256
    circle.stroke_width = stroke_width_n * 100
    circle_group.stroke_color = stroke_color
    scene_args = pydiffvg.RenderFunction.serialize_scene(\
        canvas_width, canvas_height, shapes, shape_groups)
    img = render(256,   # width
                 256,   # height
                 2,     # num_samples_x
                 2,     # num_samples_y
                 t+1,   # seed
                 None,
                 *scene_args)
    # Compute the loss function. Here it is L2.
    loss = (img - target).pow(2).sum()
    print('loss:', loss.item())

    # Backpropagate the gradients.
    loss.backward()
    # Print the gradients
    print('radius.grad:', radius_n.grad)
    print('center.grad:', center_n.grad)
    print('stroke_color.grad:', stroke_color.grad)
    print('stroke_width.grad:', stroke_width_n.grad)

    # Take a gradient descent step.
    optimizer.step()
    # Print the current params.
    print('radius:', circle.radius)
    print('center:', circle.center)
    print('stroke_width:', circle.stroke_width)
    print('stroke_color:', circle_group.stroke_color)

# Render the final result.
scene_args = pydiffvg.RenderFunction.serialize_scene(\
    canvas_width, canvas_height, shapes, shape_groups)
img = render(256,   # width
             256,   # height
             2,     # num_samples_x
             2,     # num_samples_y
             202,    # seed
             None,
             *scene_args)
# Save the images and differences.
pydiffvg.imwrite(img.cpu(), 'final.png', gamma=2.2)

Hi, thanks a lot for providing this public implementation.

I am trying to achieve a deterministic training for reproducibility.
In lines 40-41 in "painterly_rendering.py", you defined:

random.seed(1234)
torch.manual_seed(1234)

However, when running exactly the same command twice, using the same machine and environment, I get different loss values in the first iterations:
command used:
python painterly_rendering.py imgs/baboon.png --num_iter 3

output, 1:

Scene construction, time: 0.01136 s
Forward pass, time: 0.03906 s
iteration: 0
Scene construction, time: 0.00223 s
Forward pass, time: 0.00845 s
render loss: 0.2781107723712921
Backward pass, time: 0.05038 s
iteration: 1
Scene construction, time: 0.00215 s
Forward pass, time: 0.00980 s
render loss: 0.272503137588501
Backward pass, time: 0.05281 s
iteration: 2
Scene construction, time: 0.00186 s
Forward pass, time: 0.00857 s
render loss: 0.266690731048584
Backward pass, time: 0.07965 s
Scene construction, time: 0.00172 s
Forward pass, time: 0.00384 s

output, 2:

Scene construction, time: 0.02374 s
Forward pass, time: 0.01743 s
iteration: 0
Scene construction, time: 0.00133 s
Forward pass, time: 0.01063 s
render loss: 0.2781107723712921
Backward pass, time: 0.04362 s
iteration: 1
Scene construction, time: 0.00159 s
Forward pass, time: 0.00777 s
render loss: 0.2725030183792114
Backward pass, time: 0.07198 s
iteration: 2
Scene construction, time: 0.00176 s
Forward pass, time: 0.00802 s
render loss: 0.2666889429092407
Backward pass, time: 0.07049 s
Scene construction, time: 0.00244 s
Forward pass, time: 0.00385 s

you can see that in the second iteration the losses are different:
0.272503137588501 and 0.2725030183792114
Is there a way to fix that in order to achieve consistent results during training?

Thanks

Hi! I recently ran into a ModuleNotFoundError error using DiffVG on Colab. I had no trouble just last week, but for some reason, this week I get this error. Any help resolving it would be greatly appreciated!

Here is a Colab notebook to recreate the error:
https://colab.research.google.com/drive/1eGNOIl5fbZ-LoVOgODZ4cILY4wRqU9OK?usp=sharing

The import error:

ModuleNotFoundError                       Traceback (most recent call last)
[<ipython-input-1-5a80a68a63c6>](https://localhost:8080/#) in <module>()
----> 1 import pydiffvg

ModuleNotFoundError: No module named 'pydiffvg'

There was an error during insallation that may be relevant:
Traceback (most recent call last):
File "setup.py", line 98, in
zip_safe = False)
File "/usr/local/lib/python3.7/dist-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/usr/lib/python3.7/distutils/command/install_lib.py", line 109, in build
self.run_command('build_ext')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 31, in run
super().run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "setup.py", line 65, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

Attempting to install on Linux/RTXA6000.
I was only able to get the card working with newer versions of torch.

I see an error during install about cudnn 8, but then it seem to finish the install.

import throws a module missing error.

I'm very excited to work with diffvg, but I've read the install only works for old versions.
Any idea how to get past this?

diffvg_install_error.txt

Hi. I was able to install diffvg but there is an error at the import stage. I get: DLL load failed while importing diffvg: The specified module was not found.
I tried to rebuild diffvg, I tried to update python and I tried to change the whole virtual environment but the result is the same. What must be the problem? I have 10 windows and python 3.9.

The library compiles without errors under Windows (Windows 10, Python 3.8, torch 1.6, VS2019). However, running examples results in:
python single_circle.py Traceback (most recent call last): File "single_circle.py", line 1, in <module> import pydiffvg File "C:\conda\envs\NR\lib\site-packages\diffvg-0.0.1-py3.8-win-amd64.egg\pydiffvg\__init__.py", line 4, in <module> from .render_pytorch import * File "C:\conda\envs\NR\lib\site-packages\diffvg-0.0.1-py3.8-win-amd64.egg\pydiffvg\render_pytorch.py", line 18, in <module> class RenderFunction(torch.autograd.Function): File "C:\conda\envs\NR\lib\site-packages\diffvg-0.0.1-py3.8-win-amd64.egg\pydiffvg\render_pytorch.py", line 27, in RenderFunction filter = pydiffvg.PixelFilter(type = diffvg.FilterType.box, AttributeError: module 'diffvg' has no attribute 'FilterType'

dir(diffvg) ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

Any advice?

Hey there,

I tried to compile this project under Windows 10, but sadly wasn't able to do so.

At first glance, Visual Studio tries to compile the project but fails with the following error:

  diffvg.cpp
  CMake Error at diffvg_generated_diffvg.cpp.obj.Release.cmake:280 (message):
    Error generating file
    [...]/diffvg/build/temp.win-amd64-3.8/Release/CMakeFiles/diffvg.dir//Release/diffvg_generated_diffvg.cpp.obj

My enviroment is a Windows 10 with a Python 3.8.5, Annaconda 2020.11, Visual Studio 2019, Visual Studio Build Tool 2017.

Have I missed something? Do you need any more information?

Thanks in advance!

Hi! Thanks for open-sourcing this awesome work. I have one question about the differentiable renderer, if I have a batch of SVG parameters with the same dimension, can I render them in batch instead of using for loop? Thanks!

I was hoping to adapt single_path.py to Tensorflow in the vein of single_circle_tf.py. Unfortunately I have hit a roadblock.

It seems some more recent additions to the Path shape type have not been pushed over to the pydiffvg_tensorflow interface (the learnable stroke thickness and the use_distance_approx field).

I have made a feeble attempt to emulate the changes in reflected in the pydiffvg/render_pytorch.py version, but I am coming up short of a whole grokk of this massive codebase.

(The first fix was easy).

It's possible I'm 99% there, but also possible I am way off.

@BachiLi would you want to take a stab at the update? Alternatively, would you want me to push my attempt to my fork so you can perhaps try to pick up where I've left off (particularly if it's easy!)?

Thanks!

I get build diffvg and then alter site-packages\diffvg-0.0.1-py3.8-win-amd64.egg\diffvg to diffvg.pyd. And then get error with ImportError: DLL load failed while importing diffvg:

The calls to render in the Tensorflow example single_circle_tf.py (here, here, here, and here) are missing an argument.

Under the hood, it seems to be in position 8 (starting from 0), but in the python it's probably position 5. Based on another recent commit (5b65ad3), it's presumably the background image.

I attempted to resolve this in the same way the Torch examples did, by inserting a None in that position, but that led to a different error.

Edit: I think the error is probably deeper within, probably an issue with how render_tensorflow.py calls diffvg.render (i.e., here and here?).

I'd be happy to submit a PR myself to update/fix this if anyone can tell me what to insert as the appropriate place holder somewhere.

Here's the relevant stack trace (Ubuntu 18.04, CUDA 10.1, Python 3.8, Tensorflow v2.4.0-rc1):

Traceback (most recent call last):
  File "./single_circle_tf.py", line 18, in <module>
    img = render(tf.constant(256), # width
  File "/home/weinman/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/custom_gradient.py", line 261, in __call__
    return self._d(self._f, a, k)
  File "/home/weinman/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/custom_gradient.py", line 215, in decorated
    return _eager_mode_decorator(wrapped, args, kwargs)
  File "/home/weinman/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/custom_gradient.py", line 438, in _eager_mode_decorator
    result, grad_fn = f(*args, **kwargs)
  File "/home/weinman/miniconda3/lib/python3.8/site-packages/diffvg-0.0.1-py3.8-linux-x86_64.egg/pydiffvg_tensorflow/render_tensorflow.py", line 475, in render
    img, ctx = forward(width, height, num_samples_x, num_samples_y, seed, *args)
  File "/home/weinman/miniconda3/lib/python3.8/site-packages/diffvg-0.0.1-py3.8-linux-x86_64.egg/pydiffvg_tensorflow/render_tensorflow.py", line 425, in forward
    diffvg.render(scene,
TypeError: render(): incompatible function arguments. The following argument types are supported:
    1. (arg0: diffvg.Scene, arg1: diffvg.float_ptr, arg2: diffvg.float_ptr, arg3: diffvg.float_ptr, arg4: int, arg5: int, arg6: int, arg7: int, arg8: int, arg9: diffvg.float_ptr, arg10: diffvg.float_ptr, arg11: diffvg.float_ptr, arg12: diffvg.float_ptr, arg13: bool, arg14: diffvg.float_ptr, arg15: int) -> None

Invoked with: <diffvg.Scene object at 0x7f1df8bdd630>, <diffvg.float_ptr object at 0x7f1dac31b770>, <diffvg.float_ptr object at 0x7f1d846ec7f0>, <tf.Tensor: shape=(), dtype=int32, numpy=256>, <tf.Tensor: shape=(), dtype=int32, numpy=256>, 2, 2, <tf.Tensor: shape=(), dtype=int32, numpy=0>, <diffvg.float_ptr object at 0x7f1d846eccb0>, <diffvg.float_ptr object at 0x7f1d846eccf0>, <diffvg.float_ptr object at 0x7f1d846ecd30>, False

Thanks for this wonderful work!

I followed the install procedure as outlined in the docs with python 3.7. On mac it just works. On a ubuntu 18 machine with gpu (CUDA Version: 11.0), it fails with the following logs. Are the "type traits" errors the reason for failure, or something else? Been bashing my head at this for hours now and not really making any progress.

-- pybind11 v2.6.0 dev
-- Using pybind11: (version "2.6.0" dev)
-- Build with CUDA support
INFO Building without TensorFlow support (not found)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/Code/diffvg/build/temp.linux-x86_64-3.7
[ 14%] Building NVCC (Device) object CMakeFiles/diffvg.dir/diffvg_generated_scene.cpp.o
[ 28%] Building NVCC (Device) object CMakeFiles/diffvg.dir/diffvg_generated_diffvg.cpp.o
/home/ubuntu/anaconda3/envs/pytorch_latest_p37/x86_64-conda-linux-gnu/include/c++/9.3.0/type_traits(148): error: "__or_v" is not a function or static data member

/home/ubuntu/anaconda3/envs/pytorch_latest_p37/x86_64-conda-linux-gnu/include/c++/9.3.0/type_traits(148): error: "constexpr" is not valid here

/home/ubuntu/anaconda3/envs/pytorch_latest_p37/x86_64-conda-linux-gnu/include/c++/9.3.0/type_traits(150): error: "__and_v" is not a function or static data member
... more of these warnings...
/home/ubuntu/anaconda3/envs/pytorch_latest_p37/x86_64-conda-linux-gnu/include/c++/9.3.0/type_traits(2885): error: "is_scalar_v" is not a function or static data member
    self.run_command('build_ext')
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 31, in run
    super().run()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "setup.py", line 65, in build_extension
    subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

！！！I met the same error @BachiLi ，and I really need your help.
AttributeError: module 'diffvg' has no attribute 'FilterType'

I git clone the code, and pip install all requirements. Then I run python setup.py install successful.

Using c:\programdata\anaconda3\envs\torch\lib\site-packages
Finished processing dependencies for diffvg==0.0.1

But when I run single_circle.py, it showed ModuleNotFoundError: No module named 'diffvg'. So I copied all diffvg project in my conda environment site_packages. when import pydiffvg, the error occured:
File "single_circle.py", line 2, in
import pydiffvg
File "C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg_init_.py", line 4, in
from .render_pytorch import *
File "C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg\render_pytorch.py", line 21, in
class RenderFunction(torch.autograd.Function):
File "C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg\render_pytorch.py", line 30, in RenderFunction
filter=pydiffvg.PixelFilter(type=diffvg.FilterType.box,
AttributeError: module 'diffvg' has no attribute 'FilterType'

My environment：
windows10
Anaconda3
python3.7

This is such an amazing project,But I got an error after using the command 'poetry install' and 'poetry run python setup.py install' to install all python dependencies.

It is
'Current Python version (3.7.13) is not allowed by the project (=3.8).
Please change python executable via the "env use" command.'

But READEDME.md required ' Install using poetry prerequisite install python 3.7, poetry and ffmpeg'
Should I modify the current python version 3.7.13 to 3.8？ Will this affect the project?
Supplementary Note：I have successfully installed Poetry (version 1.2.0) and other prerequisite .My CUDA version is 11.1，python 3.7.13，ubantu 18.04.
I hope to get your reply, thanks!!!

What would be the steps for taking my own JPEG and generating an SVG?

Hello, I encountered a problem while using diffvg. I used a ResNet18 network added with a 3-layer MLP network to predict the control point coordinates of a path, which was rendered by diffvg to obtain the img tensor, then I use it calculate a L1 loss with my target. However, during the loss backpropagation, cuda report an error of device error, seems like some tensor needed was not sent to gpu , but if I transfer the control point coordinates to cpu, diffvg will use all tensors on cpu during rendering without this problem.

I looked through the examples in /app, I also found that some examples (GAN) also calculate loss with tensor from cpu. I’m not sure if this operation can be done on gpu?

During my debugging, I ran into this error. Does anyone know how to fix this? Thanks!

Here:

The attributes need to be passed as strings, otherwise converting the xml tree to a string in prettify() fails with:

TypeError                                 Traceback (most recent call last)
/usr/lib/python3.7/xml/etree/ElementTree.py in _escape_attrib(text)
   1079     try:
-> 1080         if "&" in text:
   1081             text = text.replace("&", "&")
TypeError: argument of type 'float' is not iterable

Here is the fix:

shape_node.set('r', str(shape.radius.item()))
shape_node.set('cx', str(shape.center[0].item()))
shape_node.set('cy', str(shape.center[1].item()))

Hi,

Diffvg appears to have issues when multiple instances of RenderFunction.apply are to be added into the computational graph.
If a second instance is called while an instance already exists under the computational graph, it will first hang and then eventually crash.

It does not output any errors to the console. It only hangs before crashing.

Here a simple script that demonstrates the behavior.
It is a simple polygon matching example, except with two keyframes.
It tries to match the two keyframes to a slowly rotating square.

import pydiffvg
import torch
import skimage
import numpy as np
import math

# Use GPU if available
pydiffvg.set_use_gpu(torch.cuda.is_available())

canvas_width, canvas_height = 256, 256

# Initialize the some target frames
targets = [];
render = pydiffvg.RenderFunction.apply

for i in range(8):
	points = torch.tensor([[128 + 60 * math.cos(0.1 * i + 0.0 * math.pi), 128 + 60 * math.sin(0.1 * i + 0.0 * math.pi)],
						   [128 + 60 * math.cos(0.1 * i + 0.5 * math.pi), 128 + 60 * math.sin(0.1 * i + 0.5 * math.pi)],
						   [128 + 60 * math.cos(0.1 * i + 1.0 * math.pi), 128 + 60 * math.sin(0.1 * i + 1.0 * math.pi)],
						   [128 + 60 * math.cos(0.1 * i + 1.5 * math.pi), 128 + 60 * math.sin(0.1 * i + 1.5 * math.pi)]])
	polygon = pydiffvg.Polygon(points = points, is_closed = True)
	shapes = [polygon]
	polygon_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor([0]),
									fill_color = torch.tensor([1.0, 1.0, 1.0, 1.0]))
	shape_groups = [polygon_group]
	scene_args = pydiffvg.RenderFunction.serialize_scene(\
					canvas_width, canvas_height, shapes, shape_groups, 
					output_type = pydiffvg.OutputType.sdf)
	img = render(256, # width
			 256, # height
			 2,   # num_samples_x
			 2,   # num_samples_y
			 0,   # seed
			 None, # background_image
			 *scene_args)
	img = img / 256
	pydiffvg.imwrite(img.cpu(), 'results/test2/target_{}.png'.format(i), gamma=2.2)
	targets.append(img.clone())

# Setup the scene
# Normalize points for easier learning rate
keyframes = []
keyframes.append(torch.tensor([[(128 + 20) / 256.0, (128 - 20) / 256.0],
							   [(128 + 20) / 256.0, (128 + 20) / 256.0],
							   [(128 - 20) / 256.0, (128 + 20) / 256.0],
							   [(128 - 20) / 256.0, (128 - 20) / 256.0]],
							  requires_grad = True))
keyframes.append(torch.tensor([[(128 + 30) / 256.0, (128 - 30) / 256.0],
							   [(128 + 30) / 256.0, (128 + 30) / 256.0],
							   [(128 - 30) / 256.0, (128 + 30) / 256.0],
							   [(128 - 30) / 256.0, (128 - 30) / 256.0]],
							  requires_grad = True))

polygon.points = keyframes[0] * 256
scene_args = pydiffvg.RenderFunction.serialize_scene(\
				canvas_width, canvas_height, shapes, shape_groups,
				output_type = pydiffvg.OutputType.sdf)
img = render(256, # width
			 256, # height
			 2,   # num_samples_x
			 2,   # num_samples_y
			 1,   # seed
			 None, # background_image
			 *scene_args)
img = img / 256
pydiffvg.imwrite(img.cpu(), 'results/test2/init.png', gamma=2.2)

# Optimizer. This is so nice. Much cleaner than in c++
optimizer = torch.optim.Adam(keyframes, lr=1e-2)

# Iterate and optimize
for t in range(100):
	print('iteration:', t)
	# Reset iteration
	optimizer.zero_grad()

	loss = torch.tensor(0.0, requires_grad = True)
	
	# Render current scene
	polygon.points = keyframes[0] * 256
	scene_args = pydiffvg.RenderFunction.serialize_scene(\
					canvas_width, canvas_height, shapes, shape_groups,
					output_type = pydiffvg.OutputType.sdf)
	img = render(256,   # width
			 	 256,   # height
			 	 2,	 # num_samples_x
			 	 2,	 # num_samples_y
			 	 t+1,   # seed
			 	 None,
			 	 *scene_args)
	img = img / 256
	pydiffvg.imwrite(img.cpu(), 'results/test2/iter_{}.png'.format(t), gamma=2.2)

	# Compute loss
	for i, tar in enumerate(targets):
		k = i / (len(targets) - 1)
		print('loss_{}'.format(k))

		polygon.points = ((1 - k) * keyframes[0] + k * keyframes[1]) * 256
		scene_args = pydiffvg.RenderFunction.serialize_scene(\
						canvas_width, canvas_height, shapes, shape_groups,
						output_type = pydiffvg.OutputType.sdf)
		print('scene_{}'.format(k))
		
		img = render(256,   # width
			 		 256,   # height
			 		 2,	 # num_samples_x
			 		 2,	 # num_samples_y
			 		 t+1,   # seed
			 		 None,
			 		 *scene_args)
		print('render_{}'.format(k))

		img = img / 256
		loss = loss + (img - tar).pow(2).sum()
		print('fin_{}'.format(k))

	print('loss:', loss.item())
	loss.backward()

	 # Take a gradient descent step.
	optimizer.step()

# Render the final result.
for i, tar in enumerate(targets):
	k = i / (len(targets) - 1)

	polygon.points = ((1 - k) * keyframes[0] + k * keyframes[1]) * 256
	scene_args = pydiffvg.RenderFunction.serialize_scene(\
					canvas_width, canvas_height, shapes, shape_groups,
					output_type = pydiffvg.OutputType.sdf)
	img = render(256,   # width
			 	 256,   # height
			 	 2,	 # num_samples_x
			 	 2,	 # num_samples_y
			 	 i+1,   # seed
			 	 None,
			 	 *scene_args)
	img = img / 256
	pydiffvg.imwrite(img.cpu(), 'results/test2/final_{}.png'.format(i), gamma=2.2)

from subprocess import call
call(["ffmpeg", "-framerate", "24", "-i",
	"results/test2/iter_%d.png", "-vb", "20M",
	"results/test2/out.mp4"])

I am using Windows 11 and a NVidia GPU.

Hi. I tried to create a circle using diffvg.
Here's my code:

import pydiffvg
import torch

canvas_width, canvas_height = 512, 512
points = torch.tensor([256, 226,
                       226, 226, 226, 256, 226, 256,
                       226, 256, 226, 286, 256, 286,
                       286, 286, 286, 256, 286, 256,
                       286, 226, 256, 226, 256, 226]).view(-1, 2)
path = pydiffvg.Path(num_control_points=torch.tensor([2, 2, 2, 2]),
                     points=points,
                     stroke_width=torch.tensor(0.5),
                     is_closed=True)
shapes = [path]
path_group = pydiffvg.ShapeGroup(shape_ids=torch.tensor([0]), fill_color=torch.tensor([0.3, 0.6, 0.3, 1.0]))
shape_groups = [path_group]
scene_args = pydiffvg.RenderFunction.serialize_scene(canvas_width, canvas_height, shapes, shape_groups)
render = pydiffvg.RenderFunction.apply
img = render(canvas_width, canvas_height, 2, 2, 0, None, *scene_args)
pydiffvg.imwrite(img.cpu(), "target.png", gamma=2.2)
target = img.clone()

But this fails with error:

Traceback (most recent call last):
  File "xxx.py", line 26, in <module>
    *scene_args)
  File "...venv\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg\render_pytorch.py", line 368, in forward
    pydiffvg.get_device().index if pydiffvg.get_device().index is not None else -1)
RuntimeError: The total length of the shape boundaries in the scene is equal or less than 0. Length = 0.000000

I don't think this is normal.

Hi,
Rendering multiple paths with variable thickness seems to be broken. Looks like it might be an issue with the bounding box computation.

Here is a code snippet to reproduce the issue:

import matplotlib.pyplot as plt
import numpy as np
import torch
import torch
import pydiffvg

pydiffvg.set_use_gpu(torch.cuda.is_available())
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
import skimage
import skimage.io

canvas_width, canvas_height = 256, 256

plt.figure(figsize=(10,5))
for num_shapes in range(1, 3):
    torch.random.manual_seed(331)

    plt.subplot(1, 2, num_shapes)
    plt.title('%d paths'%(num_shapes))
    num_control_points = torch.tensor([0, 0])
    shapes = []
    for i in range(num_shapes):
        points = torch.rand((3, 2))*canvas_width
        thickness = torch.tensor([10.0,
                                20.0,
                                30.0])
        path = pydiffvg.Path(num_control_points = num_control_points,
                            points = points,
                            is_closed = False,
                            stroke_width = thickness)
        shapes.append(path)
    path_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor(list(range(num_shapes))), #list(range(2))),
                                    fill_color = None,
                                    stroke_color = torch.tensor([0.6, 0.3, 0.6, 0.8]))
    shape_groups = [path_group]
    scene_args = pydiffvg.RenderFunction.serialize_scene(\
        canvas_width, canvas_height, shapes, shape_groups)

    render = pydiffvg.RenderFunction.apply
    img = render(256, # width
                256, # height
                2,   # num_samples_x
                2,   # num_samples_y
                0,   # seed
                None, # background_image
                *scene_args)
    plt.imshow(img.detach().cpu())
plt.show()

Hi,
Thank you so much for your contribution!
I was testing your svg_brush,py script on the note_small.svg example. I was trying to reproduce the behaviour shown in the presentation video and the brush zoom-in/zoom-out works fine. However, when I click on the svg with the brush nothing seems to happen, even though the optimization function is called.
Is there something else I have to do to make it work?

Hello! Nice work!

I'm trying to get this running on an RTX 3090, I'm getting warnings when installing where its recommending that i launch with -std=c++14

Other than that I'm not seeing anything out of the ordinary. Has anyone else managed to get this running for newer rtx cards?

  File "/opt/conda/envs/pytorch_venv/lib/python3.7/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/opt/conda/envs/pytorch_venv/lib/python3.7/site-packages/torch/autograd/__init__.py", line 149, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
  File "/opt/conda/envs/pytorch_venv/lib/python3.7/site-packages/torch/autograd/function.py", line 87, in apply
    return self._forward_cls.backward(self, *args)  # type: ignore[attr-defined]
  File "/opt/conda/envs/pytorch_venv/lib/python3.7/site-packages/diffvg-0.0.1-py3.7-linux-x86_64.egg/pydiffvg/render_pytorch.py", line 709, in backward
    eval_positions.shape[0])
RuntimeError: radix_sort: failed on 1st step: cudaErrorInvalidDevice: invalid device ordinal

when i run python setup.py install it gives the following error:
running install F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( running bdist_egg running egg_info writing diffvg.egg-info\PKG-INFO writing dependency_links to diffvg.egg-info\dependency_links.txt writing requirements to diffvg.egg-info\requires.txt writing top-level names to diffvg.egg-info\top_level.txt reading manifest file 'diffvg.egg-info\SOURCES.txt' adding license file 'LICENSE' writing manifest file 'diffvg.egg-info\SOURCES.txt' installing library code to build\bdist.win-amd64\egg running install_lib running build_py running build_ext Traceback (most recent call last): File "F:\coding_projects\python\ai_image_gen\diffvg\setup.py", line 92, in <module> setup(name = 'diffvg', File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\__init__.py", line 87, in setup return distutils.core.setup(**attrs) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup return run_commands(dist) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands dist.run_commands() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands self.run_command(cmd) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\dist.py", line 1208, in run_command super().run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command cmd_obj.run() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\install.py", line 74, in run self.do_egg_install() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\install.py", line 123, in do_egg_install self.run_command('bdist_egg') File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command self.distribution.run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\dist.py", line 1208, in run_command super().run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command cmd_obj.run() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\bdist_egg.py", line 165, in run cmd = self.call_command('install_lib', warn_dir=0) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\bdist_egg.py", line 151, in call_command self.run_command(cmdname) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command self.distribution.run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\dist.py", line 1208, in run_command super().run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command cmd_obj.run() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\install_lib.py", line 11, in run self.build() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\command\install_lib.py", line 112, in build self.run_command('build_ext') File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command self.distribution.run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\dist.py", line 1208, in run_command super().run_command(command) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command cmd_obj.run() File "F:\coding_projects\python\ai_image_gen\diffvg\setup.py", line 31, in run super().run() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\command\build_ext.py", line 84, in run _build_ext.run(self) File "F:\global_programs\Anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 346, in run self.build_extensions() File "F:\global_programs\Anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 468, in build_extensions self._build_extensions_serial() File "F:\global_programs\Anaconda3\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 494, in _build_extensions_serial self.build_extension(ext) File "F:\coding_projects\python\ai_image_gen\diffvg\setup.py", line 39, in build_extension '-DPYTHON_LIBRARY=' + get_config_var('LIBDIR'), TypeError: can only concatenate str (not "NoneType") to str

I have Anaconda version custom and python 3.9.16 installed. Kindly help.

Hello,

I really like the Idea of this project and I want to use it in personal projects, but it is not easy to use without any kind of documentation. Most of the core functionality can be reconstructed through the example code, but some at least some simple "Dos and donts" or hints to additional functionality which is not explained or used in the examples would be great. Since this project is not completely written in Python and does not have too many comments in the code, it is pretty hard do explore features that are not explicitly used in the examples. Are there plans to add more documentation?

Additionally: are there plans to publish this as a (pre compiled) package on PyPi? It is not easy to get running on windows.

Otherwise great work!

Hello @BachiLi , thanks for this awesome contribution!

I was wondering if you could give me a hint how to modify your code a bit. I was hoping to try out you painterly rendering app for sketching applications, and am not sure how to modify it to incorporate few things:

a) Remove the opacity before loss calculation / as a parameter. Sketching is done often with one pencil whose alpha is pretty much always 1.
b) An artist doing sketching usually has a constrained discrete palette to operate with (often with just black pencil / pen). It would be lovely to know how to constrain stroke colors to a specific palette (or rgb subspace)

I think I solved a) via:

         for group in shape_groups:
                group.stroke_color.data.clamp_(0.0, 1.0)
                #Limit the opactiy
                group.stroke_color.data[3].clamp_(1.0, 1.0))

Thanks in advance,

Anton.

Hi, I encountered an error when running apps/single_curve_sdf_trans.py with GPU.

~/diffvg/apps$ CUDA_VISIBLE_DEVICES=1 python single_curve_sdf_trans.py 
CUDA Runtime Error: an illegal memory access was encountered at ~/diffvg/diffvg.cpp:1631

When this error occurred, the terminal hangs.

    // Clean up weight image
    if (scene->use_gpu) {
#ifdef __CUDACC__
        checkCuda(cudaFree(weight_image)); <--------------error occurred to this line
#else
        assert(false);
#endif
    } else {
        free(weight_image);
    }

Running the installation steps on Manjaro Gnome with cuda 11.8, pytorch 12.1 only using CPU runs flawlessly. However, whenever I enable GPU (GTX 1070 Ti) compiling the setup.py installation stops at:

[ 28%] Building NVCC (Device) object CMakeFiles/diffvg.dir/diffvg_generated_diffvg.cpp.o

Here I among other errors receive the following;

CMake Error at diffvg_generated_diffvg.cpp.o.Release.cmake:280 (message):
  Error generating file
  /home/kristian/diffvg/build/temp.linux-x86_64-3.8/CMakeFiles/diffvg.dir//./diffvg_generated_diffvg.cpp.o

make[2]: *** [CMakeFiles/diffvg.dir/build.make:1103: CMakeFiles/diffvg.dir/diffvg_generated_diffvg.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:114: CMakeFiles/diffvg.dir/all] Error 2
make: *** [Makefile:103: all] Error 2
....
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

I also tried to create a small CMakeList.txt using CUDA on a small kernel example which worked correctly so I assume my CUDA is correctly setup. (Pytorch of course also recognises my GPU).

I ran DiffVG in my project and I got an error with only one line of message:
CUDA Runtime Error: operation not supported at /data/home/username/diffvg/scene.cpp:35

Here's my CUDA info returned by command nvcc -V:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

error:

File "/usr/local/lib/python3.8/dist-packages/diffvg-0.0.1-py3.8-linux-x86_64.egg/pydiffvg/render_pytorch.py", line 366, in forward
    scene = diffvg.Scene(canvas_width, canvas_height,
RuntimeError: diffvg not compiled with GPU

Values for canvas size > 32 fails at loss.backward() step running on GPU ( V100 ).

Settings:

canvas_width = 240
canvas_height = 240
num_paths = 10
num_segments = 10

Notebook freezes and nvidia-smi shows gpu memory usage drop to 0% . IPython kernel is reset.

I created a very simple notebook, it only installs the library and runs one of the examples: https://colab.research.google.com/drive/1Bnd0YI8ViGb3jDYxSGrSGmt5IcaC0GIz?usp=sharing

As you can see, the libdiffvg_tf_data_ptr_no_cxx11_abi.so is not created because the script does not find/detect Tensorflow. If you force its creation to start then a file will be created, but the error "undefined symbol: _ZTIN10tensorflow8OpKernelE" will appear.

Please provide an example of correct library installation with Tensorflow 2+ support.

Here is the error when I run python setup.py install
I am using gcc 8.2

[ 14%] Building CXX object CMakeFiles/diffvg.dir/scene.cpp.o
/scratch/arash/fast_paper/diffvg/scene.cpp: In function ‘size_t align(size_t)’:
/scratch/arash/fast_paper/diffvg/scene.cpp:13:22: error: ‘max_align_t’ is not a member of ‘std’
auto a = alignof(std::max_align_t);
^
/scratch/arash/fast_paper/diffvg/scene.cpp:13:22: note: suggested alternative:
In file included from /usr/include/c++/4.8.2/cstddef:42:0,
from /scratch/arash/fast_paper/diffvg/ptr.h:3,
from /scratch/arash/fast_paper/diffvg/color.h:5,
from /scratch/arash/fast_paper/diffvg/shape.h:4,
from /scratch/arash/fast_paper/diffvg/scene.cpp:5:
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/stddef.h:425:3: note: ‘max_align_t’
} max_align_t;
^
/scratch/arash/fast_paper/diffvg/scene.cpp:15:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
gmake[2]: *** [CMakeFiles/diffvg.dir/scene.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/diffvg.dir/all] Error 2
gmake: *** [all] Error 2
Traceback (most recent call last):
File "setup.py", line 91, in
setup(name = 'diffvg',
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 31, in run
super().run()
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "setup.py", line 65, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
File "/scratch/arash/anaconda3/envs/diffvg/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

Is diffvg able to handle 3D?

What would be involved in adapting diffvg for a 3D SDF primitive blend?

The diffvg code seems to make use of SDF for optimizing vector strokes, can the same machinery be used for composing cubes, spheres, pills etc?

In your generative_models/rendering.py you have force_cpu=True, but it works fine with all tensors on GPU (after little tweaks like cuda() insted of cpu() here and there). However, it does not give any performance boost (mnist epoch still takes 15 minutes on my machine). Why is that? Am I missing something?

Cheers,
Ivan

When running single_gradient.py, the entire process crashes with a Segmentation fault (core dumped) error.

~/diffvg/apps$ CUDA_VISIBLE_DEVICES=1 python single_gradient.py 
Segmentation fault (core dumped)

I tried to remove some part of the code and found the error was triggered by the following call to render:

render = pydiffvg.RenderFunction.apply
img = render(256, # width
             256, # height
             2,   # num_samples_x
             2,   # num_samples_y
             0,   # seed
             None, # background_image
             *scene_args)

I am using Python 3.7.9 [GCC 7.3.0] :: Anaconda, on Mac, my torch version is 1.6.0

In the single_curve.py and painterly_rendering.py examples I see that paths can be created from numpy arrays, with the points in the array representing end points and control points of a chain of quadratic bezier curves.
In the single_path.py example a cubic bezier is specified from an svg string.
However I don't see anywhere in the examples where a polyline is passed to the pydiffvg.Path() constructor.
I'd like to avoid converting my numpy array of points to an svg string and then using pydiffvg.from_svg_path().
Is this possible?
Thanks

I tried to run diffvg under PyTorch 1.8.1 with Cuda 11.1 but ran into the error:

radix_sort: failed on 1st step: cudaErrorInvalidDevice: invalid device ordinal

Some googling indicates that this might be a bug in PyTorch 1.8 and indeed downgrading to 1.7 made it go away

Environment

Windows 10 , Python Version 3.7.4

Problem

After installation using python setup install
I get this

Using c:\programdata\anaconda3\lib\site-packages
Finished processing dependencies for diffvg==0.0.1

But when running the example single circle i get this

(base) D:\GitHub\AI projects\diffvg\apps>python single_circle.py
Traceback (most recent call last):
  File "single_circle.py", line 1, in <module>
    import pydiffvg
  File "C:\ProgramData\Anaconda3\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg\__init__.py", line 4, in <module>
    from .render_pytorch import *
  File "C:\ProgramData\Anaconda3\lib\site-packages\diffvg-0.0.1-py3.7-win-amd64.egg\pydiffvg\render_pytorch.py", line 2, in <module>
    import diffvg
ModuleNotFoundError: No module named 'diffvg'

Testing if diffvg is available:

(base) D:\GitHub\AI projects\diffvg\apps>python -c "import diffvg"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'diffvg'

Also checking the package exists on the folder:

Could there be something wrong with compilation or something like that?
Also is it possible to have a compiled version?
I don't mind for which OS. hopefully for non CUDA.

In my project, I need to use some strokes that with about Bezier rank 6. I tried to do this, but it is likely that this tool just could plot stroke within Bezier rank 4.

Is there anyone could tell me how to fix it?