hologerry / attr2font

[SIGGRAPH 2020] Attribute2Font: Creating Fonts You Want From Attributes

Attr2Font

Introduction

This is the official PyTorch implementation of the Attribute2Font: Creating Fonts You Want From Attributes.

Paper: arXiv | Research Gate
Supplementary Material: link
Video: link
Code: GitHub

Abstract

Font design is now still considered as an exclusive privilege of professional designers, whose creativity is not possessed by existing software systems. Nevertheless, we also notice that most commercial font products are in fact manually designed by following specific requirements on some attributes of glyphs, such as italic, serif, cursive, width, angularity, etc. Inspired by this fact, we propose a novel model, Attribute2Font, to automatically create fonts by synthesizing visually pleasing glyph images according to user-specified attributes and their corresponding values. To the best of our knowledge, our model is the first one in the literature which is capable of generating glyph images in new font styles, instead of retrieving existing fonts, according to given values of specified font attributes. Specifically, Attribute2Font is trained to perform font style transfer between any two fonts conditioned on their attribute values. After training, our model can generate glyph images in accordance with an arbitrary set of font attribute values. Furthermore, a novel unit named Attribute Attention Module is designed to make those generated glyph images better embody the prominent font attributes. Considering that the annotations of font attribute values are extremely expensive to obtain, a semi-supervised learning scheme is also introduced to exploit a large number of unlabeled fonts. Experimental results demonstrate that our model achieves impressive performance on many tasks, such as creating glyph images in new font styles, editing existing fonts, interpolation among different fonts, etc.

Model Architecture

Demonstration

Prerequisites

• Linux
• CPU or NVIDIA GPU + CUDA cuDNN
• Python 3
• PyTorch 1.0+

Get Started

Installation

1. Install PyTorch, torchvison and dependencies from https://pytorch.org
2. Clone this repo:

   git clone https://github.com/hologerry/Attr2Font
   cd Attr2Font

3. Download the official pre-trained vgg19 model: vgg19-dcbb9e9d.pth, and put it under this project root folder

Datasets

Download the dataset from PKU Disk, Google Drive and put it into the data/:

data/
    explor_all/
        image/
        attributes.txt/

| This dataset is constructed by O’Donovan et al. Exploratory Font Selection Using Crowdsourced Attributes. TOG 2014

Model Training

python main.py --phase train

Model Testing

python main.py --phase test

Model Interpolation

python main.py --phase test_interp --test_epoch EPOCH

Citation:

If you use this code or find our work is helpful, please consider citing our work:

@article{WangSIGGRAPH2020,
  author = {Yizhi Wang*, Yue Gao*, Zhouhui Lian},
  title = {Attribute2Font: Creating Fonts You Want From Attributes},
  journal = {ACM Trans. Graph.},
  year = {2020}
}

| * Denotes equal contribution

Copyright

The code and dataset are only allowed for PERSONAL and ACADEMIC usage.

attr2font's Issues

dataloader bug

I try three times and got the same bug report

dataset download error

The dataset download link is invalidate， can you upload the dataset again?

StyleEncoder ignores the attention parameter

The StyleEncoder class takes in a boolean attention param, defaulting to true, but this param is ignored and no attention is ever used: https://github.com/hologerry/Attr2Font/blob/master/model.py#L173

sharing software showed in the demo

is it possible to get the software to explore the results?

Can you please upload the research paper for this code ?

about arguments of Test options

@hologerry

Could you explain each options meaning? (--channel, --attr_channel, --attr_embed, etc)
Actually, I want to know how to input attributes value to model.
If I want to make font which is similar with Serif font, how can I do ?

Thanks : )

Running Attr2Font on Windows

Is there some small modification or setup I can do to run this on windows?

在自建中文数据集上出现错误

感谢您出色的工作，我自己创建了一个中文数据集，运行时出现了以下错误
Traceback (most recent call last):
File "main.py", line 593, in
main()
File "main.py", line 581, in main
train(opts)
File "main.py", line 135, in train
attr_A_intensity = label_A * attr_A_data + (1 - label_A) * attr_A_embd
RuntimeError: The size of tensor a (17) must match the size of tensor b (37) at non-singleton dimension 1
出错的位置如图所示，图中两个print的输出分别为 torch.Size([64, 17])，torch.Size([64, 37])

请问您项目中的Model Architecture的图片是使用什么软件绘制的呢？

想请教一下您项目中的Model Architecture的图片是用什么软件绘制的呢？感觉非常好看，而且结构非常清晰！

Would you add the paper link and pretrained_model for download ?

train error

when input image size is 64 in style encoder，it will be transformed to a tensor which size is (1, 165, 1 ,1)
and it will be fed into residual black , which has a conv(kernel = 3) and generate an error.

Hi, Where can I download "AttrFont-CN"，I want test chinese charater like your Demostration

What is the char class for?

It looks like the generator model includes a character classifier output, which classifies the character to one of 52 one-hot values. The number 52 is hardcoded: https://github.com/hologerry/Attr2Font/blob/master/model.py#L231. What is this for? The discriminator takes in a character class param, but then ignores it and never references it https://github.com/hologerry/Attr2Font/blob/master/model.py#L367. Is the char class being in this algorithm a mistake, or is it necessary for the font generation to function? If so, why?

论文现在能提供下载么

hello! Are the attrfont CN and attrfont ENG datasets provided?

Hello. I am very interested in your experiment, and I have trained in the data set provided, and the effect is very good, so I want to further train in the Chinese data set. Is it convenient for attrfont CN and attrfont eng data set over there?

RuntimeError: CUDA error: an illegal memory access was encountered

Hi, I can not run "Sampling " part code .

Attr2Font/main.py

Lines 242 to 248 in 810404b

	val_attr_A_intensity = val_batch['attr_A'].to(device)
	val_attr_A_intensity = attr_unsuper_tolearn(val_fontembed_A)
	val_attr_A_intensity = val_attr_A_intensity.view(val_attr_A_intensity.size(0), val_attr_A_intensity.size(2))
	val_attr_A_intensity = torch.sigmoid(3*val_attr_A_intensity) # convert to [0, 1]
	val_attr_B_intensity = val_batch['attr_B'].to(device)

it occur in this part code .
And I confused L242 and L243 are the both set variable val_attr_A_intensity but with different value, why ?

Would you release the code of Demostration ？

I have a trained model ，I want test this model like your Demostration， But， it seems no code here

Model Interpolation Error

I have done Model Training and Model Testing. When I start to do Model Interpolation, the error occurs as follows:

$ python main.py --phase test_interp --test_epoch EPOCH
usage: main.py [-h] [--init_epoch INIT_EPOCH] [--n_epochs N_EPOCHS]
               [--decay_epoch DECAY_EPOCH] [--multi_gpu MULTI_GPU]
               [--data_root DATA_ROOT] [--dataset_name {explor_all}]
               [--img_size IMG_SIZE] [--batch_size BATCH_SIZE]
               [--n_threads N_THREADS] [--n_style N_STYLE] [--channel CHANNEL]
               [--attr_channel ATTR_CHANNEL] [--attr_embed ATTR_EMBED]
               [--style_out_channel STYLE_OUT_CHANNEL]
               [--n_res_blocks N_RES_BLOCKS] [--attention ATTENTION]
               [--dis_pred DIS_PRED] [--lr LR] [--b1 B1] [--b2 B2]
               [--experiment_name EXPERIMENT_NAME] [--check_freq CHECK_FREQ]
               [--sample_freq SAMPLE_FREQ] [--log_freq LOG_FREQ]
               [--phase {train,test,test_interp}] [--test_epoch TEST_EPOCH]
               [--interp_cnt INTERP_CNT] [--lambda_l1 LAMBDA_L1]
               [--lambda_char LAMBDA_CHAR] [--lambda_GAN LAMBDA_GAN]
               [--lambda_cx LAMBDA_CX] [--lambda_attr LAMBDA_ATTR]
main.py: error: argument --test_epoch: invalid int value: 'EPOCH'

Random result glyphs from generator

Hi!
Found that the generator always produces random glyphs with the same attribute parameters passed (intensity). The font as source for generation is always the same.
Is this normal behavior?
Can the generation be constant with constant input values ​​to the generator?

AttributeError: 'Namespace' object has no attribute 'selected_attrs'

Attr2Font/main.py

Line 44 in 810404b

train_dataloader = get_loader(image_dir, attribute_path, opts.selected_attrs,

used opt.selected_attrs but opt no this ！！

I'd like to know your hardware configuration and how long you take to train the whole model. Thank you very much!

This problem occurred at the beginning of training....

ValueError: Expected more than 1 spatial element when training, got input size torch.Size([32, 165, 1, 1])

I want to ask how to solve this problem,thanks

dataset download

There is some issue with the downloading of the dataset. The downloading is too slow and stops after some time. I think there is some issue on the server side.