yizhiwang96 / deepvecfont Goto Github PK
View Code? Open in Web Editor NEW[SIGGRAPH Asia 2021] DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning
License: MIT License
deepvecfont's People
Contributors
Stargazers
Watchers
Forkers
peterzhousz lianzhouhui eliheuer hzhang57 saulocatharino songpeng326 suzikim actasidiot bwtine fak3user lixinjie97 drgripa1 athena913 ymxbj minleng yayinternets jlarmstrongiv un1team-2 enyalien cv-synthesis yomismith jaedukseo coriverchen shinchu atillakorap vongillern chudotony jackzhousz ampersandio yjwang1203 microhumdeepvecfont's Issues
RuntimeError: shape '[52, 128, 128]' is invalid for input of size 212992 RuntimeError: Caught RuntimeError in DataLoader worker process 0.
I downloaded link "The Neural Rasterizer :image size 128x128"，"The Image Super-resolution model image size 128x128 -> 256x256"，"The Main model image size 128x128"，and put them in the corresponding folder，To train our main model, run python main.py --mode train --experiment_name dvf --model_name main_model , The following error appears
(dvf) miao@miao:~/data/file/bwt/deepvecfont-master$ python main.py --mode train --experiment_name dvf --model_name main_model
Training on experiment dvf_main_model...
Loading data/vecfont_dataset_pkls/train/train_all.pkl pickle file ...
Finished loading pkls
Loading data/vecfont_dataset_pkls/test/test_all.pkl pickle file ...
Finished loading pkls
Traceback (most recent call last):
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 351, in
main()
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 345, in main
train(opts)
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 320, in train
train_main_model(opts)
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 99, in train_main_model
for idx, data in enumerate(train_loader):
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/miao/data/file/bwt/deepvecfont-master/dataloader.py", line 52, in getitem
item['rendered'] = torch.FloatTensor(cur_glyph['rendered']).view(self.char_num, self.img_size, self.img_size) / 255.
RuntimeError: shape '[52, 128, 128]' is invalid for input of size 212992
The following error occurs when I change image_size default=128 in line 13 of options.py to 64
(dvf) miao@miao:~/data/file/bwt/deepvecfont-master$ python main.py --mode train --experiment_name dvf --model_name main_model
Training on experiment dvf_main_model...
Loading data/vecfont_dataset_pkls/train/train_all.pkl pickle file ...
Finished loading pkls
Loading data/vecfont_dataset_pkls/test/test_all.pkl pickle file ...
Finished loading pkls
Traceback (most recent call last):
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 351, in
main()
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 345, in main
train(opts)
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 320, in train
train_main_model(opts)
File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 64, in train_main_model
neural_rasterizer.load_state_dict(torch.load(neural_rasterizer_fpath))
File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for NeuralRasterizer:
Unexpected key(s) in state_dict: "decode.21.weight", "decode.21.bias", "decode.19.weight", "decode.19.bias".
size mismatch for decode.0.weight: copying a param with shape torch.Size([1024, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([1024, 512, 3, 3]).
size mismatch for decode.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for decode.1.weight: copying a param with shape torch.Size([1024, 2, 2]) from checkpoint, the shape in current model is torch.Size([512, 2, 2]).
size mismatch for decode.1.bias: copying a param with shape torch.Size([1024, 2, 2]) from checkpoint, the shape in current model is torch.Size([512, 2, 2]).
size mismatch for decode.3.weight: copying a param with shape torch.Size([1024, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 256, 3, 3]).
size mismatch for decode.3.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for decode.4.weight: copying a param with shape torch.Size([512, 4, 4]) from checkpoint, the shape in current model is torch.Size([256, 4, 4]).
size mismatch for decode.4.bias: copying a param with shape torch.Size([512, 4, 4]) from checkpoint, the shape in current model is torch.Size([256, 4, 4]).
size mismatch for decode.6.weight: copying a param with shape torch.Size([512, 256, 5, 5]) from checkpoint, the shape in current model is torch.Size([256, 128, 5, 5]).
size mismatch for decode.6.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for decode.7.weight: copying a param with shape torch.Size([256, 8, 8]) from checkpoint, the shape in current model is torch.Size([128, 8, 8]).
size mismatch for decode.7.bias: copying a param with shape torch.Size([256, 8, 8]) from checkpoint, the shape in current model is torch.Size([128, 8, 8]).
size mismatch for decode.9.weight: copying a param with shape torch.Size([256, 128, 5, 5]) from checkpoint, the shape in current model is torch.Size([128, 64, 5, 5]).
size mismatch for decode.9.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]).
size mismatch for decode.10.weight: copying a param with shape torch.Size([128, 16, 16]) from checkpoint, the shape in current model is torch.Size([64, 16, 16]).
size mismatch for decode.10.bias: copying a param with shape torch.Size([128, 16, 16]) from checkpoint, the shape in current model is torch.Size([64, 16, 16]).
size mismatch for decode.12.weight: copying a param with shape torch.Size([128, 64, 5, 5]) from checkpoint, the shape in current model is torch.Size([64, 32, 5, 5]).
size mismatch for decode.12.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([32]).
size mismatch for decode.13.weight: copying a param with shape torch.Size([64, 32, 32]) from checkpoint, the shape in current model is torch.Size([32, 32, 32]).
size mismatch for decode.13.bias: copying a param with shape torch.Size([64, 32, 32]) from checkpoint, the shape in current model is torch.Size([32, 32, 32]).
size mismatch for decode.15.weight: copying a param with shape torch.Size([64, 32, 5, 5]) from checkpoint, the shape in current model is torch.Size([32, 16, 5, 5]).
size mismatch for decode.15.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([16]).
size mismatch for decode.16.weight: copying a param with shape torch.Size([32, 64, 64]) from checkpoint, the shape in current model is torch.Size([16, 64, 64]).
size mismatch for decode.16.bias: copying a param with shape torch.Size([32, 64, 64]) from checkpoint, the shape in current model is torch.Size([16, 64, 64]).
size mismatch for decode.18.weight: copying a param with shape torch.Size([32, 16, 5, 5]) from checkpoint, the shape in current model is torch.Size([1, 16, 7, 7]).
size mismatch for decode.18.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([1]).
What is the problem and how can I fix it？Thank you！！
How to test the model with my own glyphs?
Hi, thank you for making this! I am trying to test the model by inputting my own SVG glyphs and with the goal of returning the results in the experiments folder. However, I can't figure out how to input my own data. From my understanding, the current test data is coming from data/vecfont_dataset_dirs/test/test_all.pkl. Do I need to format my data as a .pkl file? How can I achieve this?
AttributeError: Can't pickle local object 'get_loader.<locals>.<lambda>'
Just a note for anyone attempting to train the model on OS X and coming across this error message - due to an issue in torch (see e.g. pyg-team/pytorch_geometric#366) you will need to replace num_workers=batch_size with num_workers=0 in here:
Line 49 in e52166b
(Yes, this will seriously kill performance.)
Question abount some characters (e.g., g, p, q ...), The parts below these characters baselines are cut
The 3 SVGs generated above are g, j, and q，None of the parts below their baseline are generated.
Question :
Is there any way to make these words display normally？
If I want to get the generated font baseline information, do I need to convert the generated SVG and BMP to fonts?
No such glyph
I used ‘data/utils/convert_ttf_to_sfd_mp.py’ and add '0123456789' after charset, so the finally charset is "a-zA-Z0-9".
when convert "a-zA-Z" it is success, but it failed when char is "0"
TypeError:No such glyph
the error code is line 61 char_description.write(str(new_font_for_char[char].width) + "\n")
I check the font and it has "0" glyph in ttf file
Structure of train_all.pkl
I am using the dataset for clustering and could not understand the structure of cur_glyph['class'] in dataloader.py. I want to extract each glyph and its class labels. Can some guide me?
How can I not draw pictures through write_glyph_imgs.py
Can the generated svg sequences from the network be used to generate a font directly for use?
Can the generated svg sequences from the network be used to generate a font file (e.g., .ttf file) directly for use? Will the font size and other factors affect the visual effect in actual use?
font list used for training/testing
It seems that the font_ttfs.zip includes many fonts and only a subset of it is used for training and testing. So could you please provide the font list you used?
How to continue training on the basis of pre-trained?
Hi, thank you for making this! I loaded the parameters of the pre-trained model and trained it, But now the results more like the fonts I added
(main.py)Append code:
result
VAL/Loss
Is this an overfitting? or This is because there are no parameters to load the Adam optimizer ？
If I want to continue training my method on the basis of pre-trained, right?
A question about the NDR
Thanks for your impressive work！
however, I have a question about the Neural Differentiable Rasterizer, when you train it, why you randomly select a class of target vector glyph rather than train all of them at the same time?
I will apperciate it a lot if you can help me! Thanks a lot!
Predict on incomplete glyph set
Hi!
Is it possible to run the model against an incomplete glyph set?
For example, drawing a few glyphs myself and then having DVF generate the rest of the alphabet.
Thanks!
Not working on Apple silicon
I attempted to run DeepVecFont on my M1 MacBook Pro, but the software is only compatible with CUDA. Are there any plans to release a version that can run on the CPU?
Thank you very much,
Marco
License?
The readme didn't state a license and there's no license file :) please clarify
An error occurs using my own mean.npz and stdev.npz
I've customized my own dataset and I'm getting errors loading mean.npz and stdev.npz when running main.py. I found out that the type of the loaded data was numpy.lib.npyio.NpzFile. The mean and stdev files downloaded from Google drive seem to be actually .npy format, so the problem does not occur. write_data_to_pkl.py saves them by np.savez, so for example it seems that the load side can support both formats if you do as follows:
@@ -89,9 +89,14 @@ def train_main_model(opts):
if opts.tboard:
writer = SummaryWriter(log_dir)
-
- mean = np.load('./data/mean.npz')
- std = np.load('./data/stdev.npz')
+
+ def extract_npz_if_needed(data):
+ if not isinstance(data, np.ndarray):
+ data = data[data.files[0]]
+ return data
+
+ mean = extract_npz_if_needed(np.load('./data/mean.npz'))
+ std = extract_npz_if_needed(np.load('./data/stdev.npz'))
mean = torch.from_numpy(mean).to(device).to(torch.float32)
std = torch.from_numpy(std).to(device).to(torch.float32)
network_modules= [img_encoder, img_decoder, modality_fusion, vggptlossfunc, svg_encoder, svg_decoder, mdn_top_layer, neural_rasterizer]Few-shot font generation via different `ref_nshot`
Hi, there,
The checkpoints of pre-trained model use ref_nshot=4 for training and testing.
However, it seems like if I want to use different ref_nshot value (for example, 6, for ABCabc), I need to train the model from beginning rather than using the pre-trained model directly, or it will show dimension unmatch error during test:
RuntimeError: Error(s) in loading state_dict for ModalityFusion:
size mismatch for seq_fc.weight: copying a param with shape torch.Size([512, 4096]) from checkpoint, the shape in current model is torch.Size([512, 6144]).
Is there any more efficiency way to test the performance on different ref_nshot, such as training ModalityFusion only, rather than retrained the model everytime?
Thank you.
OSError: [WinError 1455] and memory error
- When I am training the main module, I get OSError: [WinError 1455] and memory error, can you please provide me with what I need to train the model?
- Is the current program input data only supports svg, does it support bitmap?
- What directory should Input glyphs be placed in the demo, what is the program run command, and in which directory will the fonts be generated by deepvecfont
generate random fonts
How do you run your code if you want to generate random fonts
Code error
Using DeepVecFont for icon generation
Hi,
Thank you for the great work and for releasing your code for public use. The current preprocessor converts fonts (ttf files) to sfd format and then maps that to a restricted svg format for training the model. If we already have the svg files instead of the ttf files (e.g. for icons) and want to convert that to the same simplified svg format, in order to use the model to train on icons, could you please provide some guidance on the preprocessing needed for this direct conversion from svg to the simplified svg needed for the model?
thanks
About the main model
Hi,
Thank you for releasing the amazing repo. Could you tell me when you'll upload the main model?
Thank you again.
Sincerely yours.
The command length for SVG <=50
Hi,Does the model only support fonts with a length SVG command length of less than 50? I found that the is_valid_path function of the svg_util is called on line 71 of the write_data_to_dirs.py file,An SVG command is only valid if the length < = 50
I'd like to change this to 50 instead of 100, 200, 300 or 1000, is that okay? Was it set to 50 because it worked best when it was set to 50?
The MDN loss drops to negative values during training.
Has this happened to you during your experiments? Is this a normal occurrence?
Some fonts may not be processed by convert_ttf_to_sfd_mp.py
Hi, I'm encountering the problem in the title.
For example, if mp.cpu_count()=8 and you have 8 fonts under font_ttfs/test, one font will not be processed.
In this case, process_nums=6 and font_num_per_process=1.
Processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)].will launch 7 processes, but they will each process one font, so only 7 fonts will be processed in total.
It's not a pretty solution, but it seems to work by having the process with the largest pid handle all the remaining fonts, as shown below:
--- a/data_utils/convert_ttf_to_sfd_mp.py
+++ b/data_utils/convert_ttf_to_sfd_mp.py
@@ -26,8 +26,8 @@ def convert_mp(opts):
else:
font_num_per_process = font_num // process_nums
- def process(process_id, font_num_p_process):
- for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process):
+ def process(process_id, font_num_p_process, last=False):
+ for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process if not last else font_num):
if i >= font_num:
break
@@ -72,7 +72,7 @@ def convert_mp(opts):
cur_font.close()
- processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)]
+ processes = [mp.Process(target=process, args=(pid, font_num_per_process, pid==process_nums)) for pid in range(process_nums + 1)]
for p in processes:
p.start()About installing Diffvg
For anyone who installed diffvg before 2021.10.19:
I forgot to mention that we added a new function: save_svg_paths_only in diffvg/pydiffvg/save_svg.py.
You need first replace the original diffvg/pydiffvg/save_svg.py with this and then reinstall.
About the SVG-VAE dataset
Hi! Thank you for following this. I downloaded the glyphazzn_urls.txt files in the SVG-VAE project and ran it download_ttf_fonts.py only downloaded more than 4000 fonts,what should I do if I want to download the 14M fonts mentioned in the paper?
data missing
Could you provide the extra training data for image superresolution and neural rasterizer? The missing data seem to includes./data/mean.npz, ./data/stdev.npz and ./data/glyphss_dataset. Thanks!
An error occurs when running train_nr.py
In train_nr.py#L35 you pass an argument named n_blocks, but neural_rasterizer.py shows that an argument with this name does not exist in NeuralRasterizer.__init__'s arguments. Should the value be passed to n_upsampling?
input
Is your trained model only able to enter 52 letters when tested? You get an error if it's not 52
train the image super-resolution model error and train our main model error
手顺
代码版本 3ba4adb
最后更新时间: Fri Dec 31 12:10:07 2021 +0800
使用命令将ttf/otf文件做成数据集
使用命令:
> cd data_utils
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split train
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split test
> python write_glyph_imgs.py --split train
> python write_glyph_imgs.py --split test
> python write_glyph_imgs.py --split train --img_size=256
> python write_glyph_imgs.py --split test --img_size=256
> python write_data_to_pkl.py --split train
> python write_data_to_pkl.py --split test
步骤2:train the neural rasterizer:
使用命令
mv ./data/vecfont_dataset_/ ./data/vecfont_dataset/
mv ./data/vecfont_dataset/train/mean.npz ./data/vecfont_dataset/train/stdev.npz ./data
python train_nr.py --mode train --experiment_name dvf --model_name neural_raster
步骤3. train the image super-resolution model:
使用命令:
cp -r ./data/vecfont_dataset ./data/glyphss_dataset
python train_sr.py --mode train --name image_sr
出现下面错误
----------------- Options ---------------
batch_size: 2
beta1: 0.0
char_categories: 52
checkpoints_dir: ./experiments
continue_train: False
crop_size: 256
dataroot: ./data/glyphss_dataset/
dataset_mode: aligned
direction: BtoA
display_env: main
display_freq: 400
display_id: 1
display_ncols: 4
display_port: 8097
display_server: http://172.31.222.102
display_winsize: 256
epoch: latest
epoch_count: 1
experiment_name: dvf
gan_mode: lsgan
gauss_temperature: 0
gpu_ids: 0
image_size: 256
init_gain: 0.02
init_type: normal
input_nc: 1
isTrain: True [default: None]
lambda_L1: 1.0
load_iter: 0 [default: 0]
load_size: 256
lr: 0.002
lr_decay_iters: 50
lr_policy: linear
max_dataset_size: inf
mix_temperature: 0.0001
mode: train
model_name: main_model
n_epochs: 500
n_epochs_decay: 500
n_layers_D: 3
name: image_sr
ndf: 64
netD: basic
netG: unet_256
ngf: 64
no_dropout: False
no_flip: True
no_html: False
norm: instance
num_threads: 4
output_nc: 1
phase: train
pool_size: 50
preprocess: none
print_freq: 100
save_by_iter: False
save_epoch_freq: 25
save_latest_freq: 5000
serial_batches: False
suffix:
test_epoch: 125
update_html_freq: 1000
verbose: False
----------------- End -------------------
Loading ./data/glyphss_dataset/train/train_all.pkl pickle file ...
Finished loading
Loading ./data/glyphss_dataset/test/test_all.pkl pickle file ...
Finished loading
initialize network with normal
initialize network with normal
model [Pix2PixModel] was created
---------- Networks initialized -------------
[Network G] Total number of parameters : 54.403 M
[Network D] Total number of parameters : 2.764 M
-----------------------------------------------
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
learning rate 0.0020000 -> 0.0020000
Traceback (most recent call last):
File "/home/ubuntu/deepvecfont/train_sr.py", line 29, in <module>
for i, data in enumerate(dataset_train): # inner loop within one epoch
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 652, in __next__
data = self._next_data()
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1347, in _next_data
return self._process_data(data)
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1373, in _process_data
data.reraise()
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
raise exception
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/deepvecfont/dataloader_sr.py", line 30, in __getitem__
item['rendered_256'] = torch.FloatTensor(cur_glyph['rendered_256']).view(self.char_num, 256, 256) / 255.
KeyError: 'rendered_256'
修改错误
在data_util/write_data_to_pkl.py文件的93行追加以下代码,解决错误
```
93 if not os.path.exists(os.path.join(cur_font_sfd_dir, 'imgs_256.npy')):
94 rendered256 = np.zeros((opts.num_char, opts.img_size, opts.img_size), np.uint8)
95 rendered256[:, :, :] = 255
96 rendered256 = rendered.tolist()
97 else:
98 rendered256 = np.load(os.path.join(cur_font_sfd_dir, 'imgs_256.npy')).tolist()
99 merged_res['rendered_256'] = rendered256
```
步骤4: train our main model
python main.py --mode train --experiment_name dvf --model_name main_model
发生错误:
Epoch: 899/2000, Batch: 0/1, Loss: 0.913249, img_l1_loss: 0.168880, kl_loss: 0.003375, img_pt_c_loss: 0.020982, mdn_loss: 0.461596, softmax_xent_loss: 0.073407, synsvg_nr_recloss: 0.185008
Epoch: 949/2000, Batch: 0/1, Loss: 0.678629, img_l1_loss: 0.123693, kl_loss: 0.006045, img_pt_c_loss: 0.020730, mdn_loss: 0.283389, softmax_xent_loss: 0.115898, synsvg_nr_recloss: 0.128875
Epoch: 999/2000, Batch: 0/1, Loss: 0.780886, img_l1_loss: 0.189140, kl_loss: 0.004557, img_pt_c_loss: 0.024209, mdn_loss: 0.270104, softmax_xent_loss: 0.115126, synsvg_nr_recloss: 0.177751
Traceback (most recent call last):
File "/home/ubuntu/deepvecfont/main.py", line 358, in <module>
main()
File "/home/ubuntu/deepvecfont/main.py", line 352, in main
train(opts)
File "/home/ubuntu/deepvecfont/main.py", line 327, in train
train_main_model(opts)
File "/home/ubuntu/deepvecfont/main.py", line 167, in train_main_model
val_img_decoder_out, val_vggpt_loss, val_kl_loss, val_svg_losses, val_trg_img, val_ref_img, val_trgsvg_nr_out, val_synsvg_nr_out = network_forward(val_data, mean, std, opts, network_modules)
File "/home/ubuntu/deepvecfont/main.py", line 318, in network_forward
svg_losses = mdn_top_layer.svg_loss(top_output, trg_seq, trg_seqlen+1, opts.max_seq_len)
File "/home/ubuntu/deepvecfont/models/svg_decoder.py", line 185, in svg_loss
seqlen_mask = util_funcs.sequence_mask(trg_seqlen, max_seq_len)
File "/home/ubuntu/deepvecfont/models/util_funcs.py", line 80, in sequence_mask
.lt(lengths.unsqueeze(1)))
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
问题
- 步骤2中发生了错误, 我修改了data_util/write_data_to_pkl.py中的代码，追加代码如下图所示，这样修改代码是否存在问题?
- 步骤4中发生的错误，是做成的数据有问题吗？还是其他原因?可以帮忙看一下吗？
Questions about the glyph images
Hi, I found that some glyph images in ground truth are missing some of the handwriting. Where did i go wrong ？
Random Sampling From Latent Space
Hi! Can you share code for Random Sampling From Latent Space experiment?
train the image super-resolution model error and train the image super-resolution model error
手顺
- 代码版本 : e3bd2e
- 最后更新时间: Date: Sat Jul 23 22:53:13 2022 +0800
使用命令将ttf/otf文件做成数据集
使用命令:
> cd data_utils
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split train
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split test
> python write_glyph_imgs.py --split train
> python write_glyph_imgs.py --split test
> python write_data_to_dirs.py --split train
> python write_data_to_dirs.py --split test
步骤2:train the neural rasterizer:
使用命令:
- mv ./data/vecfont_dataset_dirs_/ ./data/vecfont_dataset_dirs
- python train_nr.py --mode train --experiment_name dvf --model_name neural_raster
发生下面错误:
Training on experiment dvf_neural_raster...
Finished loading train paths
Traceback (most recent call last):
File "/home/ubuntu/deepvecfontnew/train_nr.py", line 231, in <module>
main()
File "/home/ubuntu/deepvecfontnew/train_nr.py", line 223, in main
train(opts)
File "/home/ubuntu/deepvecfontnew/train_nr.py", line 191, in train
train_nr_model(opts)
File "/home/ubuntu/deepvecfontnew/train_nr.py", line 29, in train_nr_model
train_loader = get_loader(opts.data_root, opts.image_size, opts.char_categories, opts.max_seq_len, opts.seq_feature_dim, opts.batch_size, opts.read_mode, opts.mode)
File "/home/ubuntu/deepvecfontnew/dataloader.py", line 69, in get_loader
dataloader = data.DataLoader(dataset, batch_size, shuffle=(mode == 'train'), num_workers=batch_size, drop_last=True)
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 347, in __init__
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 107, in __init__
raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0
修改代码
将文件 dataloader.py 69的shuffle=(mode == 'train')修改为shuffle=False
继续使用命令:python train_nr.py --mode train --experiment_name dvf --model_name neural_raster
在train_nr.py 的57行加入 print(len(train_loader))
输出一下log:
```
Training on experiment dvf_neural_raster...
Finished loading train paths
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 64 worker processes in total. Our suggested max number of worker in current system is 12, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
Finished loading test paths
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
warnings.warn(
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=VGG19_Weights. IMAGENET1K_V1`. You can also use `weights=VGG19_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
0
0
0
0
0
0
0
0
```
显示len(train_loader)的值为0?这是因为生成的数据有问题吗？还是因为我修改的代码导致的原因。
步骤3. train the image super-resolution model:
使用命令:
python train_sr.py --mode train --name image_sr
发生下面错误:
----------------- Options ---------------
batch_size: 16
beta1: 0.0
char_categories: 52
checkpoints_dir: ./experiments
continue_train: False
crop_size: 256
dataroot: ./data/vecfont_dataset_dirs_sr/
dataset_mode: aligned
direction: BtoA
display_env: main
display_freq: 400
display_id: 1
display_ncols: 4
display_port: 8097
display_server: http://172.31.222.102
display_winsize: 256
epoch: latest
epoch_count: 1
experiment_name: dvf
gan_mode: lsgan
gauss_temperature: 0
gpu_ids: 0
image_size: 256
img_hr: 256
img_lr: 128
init_gain: 0.02
init_type: normal
input_nc: 1
isTrain: True [default: None]
lambda_L1: 1.0
load_iter: 0 [default: 0]
load_size: 256
lr: 0.002
lr_decay_iters: 50
lr_policy: linear
max_dataset_size: inf
mix_temperature: 0.0001
mode: train
model_name: main_model
n_epochs: 500
n_epochs_decay: 500
n_layers_D: 3
name: image_sr
ndf: 64
netD: basic
netG: unet_256
ngf: 64
no_dropout: False
no_flip: True
no_html: False
norm: instance
num_threads: 4
output_nc: 1
phase: train
pool_size: 50
preprocess: none
print_freq: 100
read_mode: dirs
save_by_iter: False
save_epoch_freq: 25
save_latest_freq: 5000
serial_batches: False
suffix:
test_epoch: 125
update_html_freq: 1000
verbose: False
----------------- End -------------------
Traceback (most recent call last):
File "/home/ubuntu/deepvecfontnew/train_sr.py", line 12, in <module>
dataset_train, dataset_test = create_dataset(opt) # create a dataset given opt.dataset_mode and other options
File "/home/ubuntu/deepvecfontnew/models/imgsr/modules.py", line 244, in create_dataset
dataset_train = get_loader(opt.dataroot, opt.char_categories, opt.batch_size, opt.read_mode, opt.img_lr, opt.img_hr, 'train')
File "/home/ubuntu/deepvecfontnew/dataloader_sr.py", line 63, in get_loader
dataloader = data.DataLoader(dataset, batch_size, shuffle=(mode == 'train'), num_workers=batch_size)
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 347, in __init__
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 107, in __init__
raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0
修改代码解决错误
将dataloader_sr.py文件63行的shuffle=(mode == 'train')=> shuffle=False
再次运行步骤4发生以下错误:
home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
learning rate 0.0020000 -> 0.0020000
End of epoch 1 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 2 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 3 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 4 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 5 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 6 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 7 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 8 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 9 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 10 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 11 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 12 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 13 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 14 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 15 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 16 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 17 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 18 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 19 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 20 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 21 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 22 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 23 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
End of epoch 24 / 1000 Time Taken: 0 sec
learning rate 0.0020000 -> 0.0020000
Traceback (most recent call last):
File "/home/ubuntu/deepvecfontnew/train_sr.py", line 45, in <module>
writer.add_scalar('TRAIN/l1_loss', criterionL1(model.real_B, model.fake_B), epoch)
AttributeError: 'Pix2PixModel' object has no attribute 'real_B'
问题
- 步骤2中将文件 dataloader.py 69行的shuffle=(mode == 'train')修改为shuffle=False，是否会存在问题?
- 步骤2中在train_nr.py 的57行加入 print(len(train_loader))，输出的结果为0，是因为数据有问题吗？还是其他原因?
- 步骤3出现的错误AttributeError: 'Pix2PixModel' object has no attribute 'real_B'，这是什么原因?
Some questions about NDR
This is such an amazing project！
How did you pre-train the NDR model? What are the labels for the training set and the test set?Can the NDR you provide be used to generate vector English characters for each style? Do I need to retrain NDR for each style English Fonts?
How to train my own NDR for icons or Chinese characteres？What are the labels for the training set and the test set?
I hope to get your reply, thanks!!!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.