Comments (12)
@DYF-AI it should work if you change color[k]
to self.colors[k]
on line 281.
from im2vec.
/Im2Vec/logs/VectorVAEnLayers/version_110/models/vector_vae_nlayers.py", line 281, in save
color = make_tensor(color[k])
UnboundLocalError: local variable 'color' referenced before assignment
from im2vec.
Hey @DYF-AI
I have few deadlines to work on this week so I'll look into the issue next week.
Meanwhile heres the code I used to generate the images for the paper.
https://gist.github.com/preddy5/29db261676188dd075b95ebf24b2cd19
use it like this
save_svg(f"{save_dir}/{name}.svg", self.imsize, self.imsize, shapes, shape_groups)
shapes and shape_groups are the variables in the raster function here https://github.com/preddy5/Im2Vec/blob/master/models/vector_vae.py#L266
Regards,
Pradyumna.
from im2vec.
@DYF-AI it should work if you change
color[k]
toself.colors[k]
on line 281.
(1) I change expriment.py in line 252
if save_svg:
self.model.save(test_input, save_dir, name) and run CUDA_VISIBLE_DEVICES=1 python3 eval_local.py -c configs/emoji.yaml
/Im2Vec/logs/VectorVAEnLayers/version_110/models/vector_vae_nlayers.py", line 281, in save
color = make_tensor(self.colors[k])
IndexError: list index out of range
from im2vec.
@DYF-AI it should work if you change
color[k]
toself.colors[k]
on line 281.(1) I change expriment.py in line 252
if save_svg:
self.model.save(test_input, save_dir, name) and run CUDA_VISIBLE_DEVICES=1 python3 eval_local.py -c configs/emoji.yaml/Im2Vec/logs/VectorVAEnLayers/version_110/models/vector_vae_nlayers.py", line 281, in save
color = make_tensor(self.colors[k])
IndexError: list index out of range
Have you solved this problem and save .svg files ?
from im2vec.
Hey @DYF-AI
I have few deadlines to work on this week so I'll look into the issue next week.
Meanwhile heres the code I used to generate the images for the paper.https://gist.github.com/preddy5/29db261676188dd075b95ebf24b2cd19
use it like this
save_svg(f"{save_dir}/{name}.svg", self.imsize, self.imsize, shapes, shape_groups)shapes and shape_groups are the variables in the raster function here https://github.com/preddy5/Im2Vec/blob/master/models/vector_vae.py#L266
Regards,
Pradyumna.
When I use this code to generator svg file ,
points = all_points[k].cpu().contiguous()#[self.sort_idx[k]] print(points.shape) print(points[4, 0])
IndexError: index 4 is out of bounds for dimension 0 with size 3
The input is input tensor ,can you provide a png to svg demo in your work, thanks ~
from im2vec.
Hey @kelisiya
Could you elaborate more on how you used the gist function I shared.
from im2vec.
Hey @kelisiya
Could you elaborate more on how you used the gist function I shared.
First, I use the eval_local function to load the weights to predict the default data/train, and at the same time I put the save_svg function in vector_vae_nlayers.py to replace it. If I send the output result to save_svg, such an error will be reported. I guess it is because the output of the model is a comprehensive graph rather than a single result?
from im2vec.
@kelisiya sorry it looks like my suggestions were confusing, here https://gist.github.com/preddy5/a666d758674300366c280a2bbfe5c5ad this is the notebook I use for generating some results for the paper. Disclaimer: A lot of the code in there is not optimized or ready for release, but it should help you out.
from im2vec.
@kelisiya sorry it looks like my suggestions were confusing, here https://gist.github.com/preddy5/a666d758674300366c280a2bbfe5c5ad this is the notebook I use for generating some results for the paper. Disclaimer: A lot of the code in there is not optimized or ready for release, but it should help you out.
thank you very much ~ I can run the demo , if I want to draw my svg , I should train the new model ? I selet some png to data file and run the inference , I found the result error . For example , I use airplane.png to data file , the result still is emoji_u1f632's result .
from im2vec.
Using the save_svg code directly did not generate the desired output even after fixing the indexing errors mentioned above. I think the key is to figure out how to generate the points from the reconstructed image. Here's an alternative solution that I used.
The code in painterly_rendering.py at https://github.com/BachiLi/diffvg/blob/master/apps/painterly_rendering.py generates svg output from an image input. It uses backprop to iteratively generate an SVG output that matches the input. I used this with the code in sample_interpolate() in experiment.py to generate a SVG output for example, for the first reconstructed image (recons[0]) as follows:
if save_svg:
#self.model.save(test_input, save_dir, name) #original code
genSVG(recons[0], save_dir)
The following is the code for genSVG based on the code in painterly_rendering.py. The original code loads the input image from a file. I have modified it to take as input an image generated by the VectorVAEnLayers model (used in experiment.py).
This code generates an SVG output periodically during the optimization (to minimize the LPIPS loss) and you can specify the number of iterations (num_iter) to achieve the desired output.
import pydiffvg
import torch
import skimage
import skimage.io
import random
import ttools.modules
import math
pydiffvg.set_print_timing(True)
gamma = 1.0
def genSVG(inp_img, save_dir):
num_paths = 512
max_width = 2.0
use_lpips_loss = True
num_iter = 500
use_blob = False
# Use GPU if available
pydiffvg.set_use_gpu(torch.cuda.is_available())
perception_loss = ttools.modules.LPIPS().to(pydiffvg.get_device())
#target = torch.from_numpy(skimage.io.imread('imgs/lena.png')).to(torch.float32) / 255.0
#target = torch.from_numpy(inp_img).to(torch.float32) / 255.0
target = inp_img.to(pydiffvg.get_device())
target = target.pow(gamma)
target = target.unsqueeze(0)
print(target.shape, target)
#target = target.permute(0, 3, 1, 2) # NHWC -> NCHW
#target = torch.nn.functional.interpolate(target, size = [256, 256], mode = 'area')
canvas_width, canvas_height = target.shape[3], target.shape[2]
num_paths = num_paths
max_width = max_width
random.seed(1234)
torch.manual_seed(1234)
shapes = []
shape_groups = []
if use_blob:
for i in range(num_paths):
num_segments = random.randint(3, 5)
num_control_points = torch.zeros(num_segments, dtype = torch.int32) + 2
points = []
p0 = (random.random(), random.random())
points.append(p0)
for j in range(num_segments):
radius = 0.05
p1 = (p0[0] + radius * (random.random() - 0.5), p0[1] + radius * (random.random() - 0.5))
p2 = (p1[0] + radius * (random.random() - 0.5), p1[1] + radius * (random.random() - 0.5))
p3 = (p2[0] + radius * (random.random() - 0.5), p2[1] + radius * (random.random() - 0.5))
points.append(p1)
points.append(p2)
if j < num_segments - 1:
points.append(p3)
p0 = p3
points = torch.tensor(points)
points[:, 0] *= canvas_width
points[:, 1] *= canvas_height
path = pydiffvg.Path(num_control_points = num_control_points,
points = points,
stroke_width = torch.tensor(1.0),
is_closed = True)
shapes.append(path)
path_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor([len(shapes) - 1]),
fill_color = torch.tensor([random.random(),
random.random(),
random.random(),
random.random()]))
shape_groups.append(path_group)
else:
for i in range(num_paths):
num_segments = random.randint(1, 3)
num_control_points = torch.zeros(num_segments, dtype = torch.int32) + 2
points = []
p0 = (random.random(), random.random())
points.append(p0)
for j in range(num_segments):
radius = 0.05
p1 = (p0[0] + radius * (random.random() - 0.5), p0[1] + radius * (random.random() - 0.5))
p2 = (p1[0] + radius * (random.random() - 0.5), p1[1] + radius * (random.random() - 0.5))
p3 = (p2[0] + radius * (random.random() - 0.5), p2[1] + radius * (random.random() - 0.5))
points.append(p1)
points.append(p2)
points.append(p3)
p0 = p3
points = torch.tensor(points)
points[:, 0] *= canvas_width
points[:, 1] *= canvas_height
#points = torch.rand(3 * num_segments + 1, 2) * min(canvas_width, canvas_height)
path = pydiffvg.Path(num_control_points = num_control_points,
points = points,
stroke_width = torch.tensor(1.0),
is_closed = False)
shapes.append(path)
path_group = pydiffvg.ShapeGroup(shape_ids = torch.tensor([len(shapes) - 1]),
fill_color = None,
stroke_color = torch.tensor([random.random(),
random.random(),
random.random(),
random.random()]))
shape_groups.append(path_group)
scene_args = pydiffvg.RenderFunction.serialize_scene(\
canvas_width, canvas_height, shapes, shape_groups)
render = pydiffvg.RenderFunction.apply
img = render(canvas_width, # width
canvas_height, # height
2, # num_samples_x
2, # num_samples_y
0, # seed
None,
*scene_args)
pydiffvg.imwrite(img.cpu(), save_dir+'init.png', gamma=gamma)
points_vars = []
stroke_width_vars = []
color_vars = []
for path in shapes:
path.points.requires_grad = True
points_vars.append(path.points)
if not use_blob:
for path in shapes:
path.stroke_width.requires_grad = True
stroke_width_vars.append(path.stroke_width)
if use_blob:
for group in shape_groups:
group.fill_color.requires_grad = True
color_vars.append(group.fill_color)
else:
for group in shape_groups:
group.stroke_color.requires_grad = True
color_vars.append(group.stroke_color)
# Optimize
points_optim = torch.optim.Adam(points_vars, lr=1.0)
if len(stroke_width_vars) > 0:
width_optim = torch.optim.Adam(stroke_width_vars, lr=0.1)
color_optim = torch.optim.Adam(color_vars, lr=0.01)
# Adam iterations.
for t in range(num_iter):
print('iteration:', t)
points_optim.zero_grad()
if len(stroke_width_vars) > 0:
width_optim.zero_grad()
color_optim.zero_grad()
# Forward pass: render the image.
scene_args = pydiffvg.RenderFunction.serialize_scene(\
canvas_width, canvas_height, shapes, shape_groups)
img = render(canvas_width, # width
canvas_height, # height
2, # num_samples_x
2, # num_samples_y
t, # seed
None,
*scene_args)
# Compose img with white background
img = img[:, :, 3:4] * img[:, :, :3] + torch.ones(img.shape[0], img.shape[1], 3, device = pydiffvg.get_device()) * (1 - img[:, :, 3:4])
# Save the intermediate render.
pydiffvg.imwrite(img.cpu(), save_dir+'/iter_{}.png'.format(t), gamma=gamma)
img = img[:, :, :3]
# Convert img from HWC to NCHW
img = img.unsqueeze(0)
img = img.permute(0, 3, 1, 2) # NHWC -> NCHW
if use_lpips_loss:
loss = perception_loss(img, target) + (img.mean() - target.mean()).pow(2)
else:
loss = (img - target).pow(2).mean()
print('render loss:', loss.item())
# Backpropagate the gradients.
loss.backward()
# Take a gradient descent step.
points_optim.step()
if len(stroke_width_vars) > 0:
width_optim.step()
color_optim.step()
if len(stroke_width_vars) > 0:
for path in shapes:
path.stroke_width.data.clamp_(1.0, max_width)
if use_blob:
for group in shape_groups:
group.fill_color.data.clamp_(0.0, 1.0)
else:
for group in shape_groups:
group.stroke_color.data.clamp_(0.0, 1.0)
if t % 10 == 0 or t == num_iter - 1:
pydiffvg.save_svg(save_dir+'iter_{}.svg'.format(t),
canvas_width, canvas_height, shapes, shape_groups)
# Render the final result.
img = render(canvas_width, #target.shape[1], # width
canvas_height, #target.shape[0], # height
2, # num_samples_x
2, # num_samples_y
0, # seed
None,
*scene_args)
# Save the intermediate render.
pydiffvg.imwrite(img.cpu(), save_dir+'final.png'.format(t), gamma=gamma)
# Convert the intermediate renderings to a video.
from subprocess import call
call(["ffmpeg", "-framerate", "24", "-i",
save_dir+"iter_%d.png", "-vb", "20M",
save_dir+"out.mp4"])
from im2vec.
I apologize I didnt realize that the python notebook URL is throwing an error, here's a better one https://gist.github.com/preddy5/68bf5cf78a147cb4ae58d6698d095f93
Hey @athena913 thank you for the suggestion, yes the code you share should give you a vector graphic output however I don't think that it is the most optimal solution since it would give you a vector graphic with 512 paths, instead if you save the vectors estimated by the im2vec decoder in the case of emojis the same image would be represented with 4paths.
from im2vec.
Related Issues (20)
- the loss is NAN when train model HOT 7
- problems when training HOT 8
- training on real images?
- Default config (emoji.yaml) fails to converge HOT 7
- Why are there many copies in the dataset? HOT 2
- Post-decoding process HOT 3
- emoji dataset is settled to 4 parts, and every parts is filled with a constant color, does this mean im2vec can only deal with HOT 5
- Segmentation tool HOT 4
- Speeding up training and using attention HOT 1
- Chamfer distance code
- Can't install dependencies HOT 1
- Training example from README wont work
- Dependency Conflict with Pillow
- MisconfigurationException in Training Example
- pip install -r requirements.txt. ERRORS HOT 1
- About training details HOT 6
- RuntimeError: stack expects a non-empty TensorList HOT 3
- The difference between VectorVAE and VectorVAEnLayers
- No module named 'pydiffvg' HOT 1
- How to train with celeba dataset? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from im2vec.