A collection of papers about Sketch Synthesis (Generation). Mainly focus on stroke-level vector sketch synthesis.
Feel free to create a PR or an issue.
Outlines
Here Vector strokes
means having svg data. With photos
means having the photo-sketch paired data.
Level | Dataset | Source | Vector strokes | With photos | Notes |
Characters | Omniglot | ✔️ | ❌ | Alphabets characters | |
KanjiVG | ✔️ | ❌ | Chinese characters | ||
Kuzushiji | ❌ | ❌ | Japanese characters | ||
Instance-level | TU-Berlin | SIGGRAPH 2012 | ✔️ | ❌ | Multi-category hand sketches |
Sketchy | SIGGRAPH 2016 | ✔️ | ✔️ | Multi-category photo-sketch paired | |
QuickDraw | ICLR 2018 | ✔️ | ❌ | Multi-category hand sketches | |
QMUL-Shoe-Chair-V2 | CVPR 2016 | ✔️ | ✔️ | Only two categories | |
Sketch Perceptual Grouping (SPG) | ECCV 2018 | ✔️ | ❌ | With part-level semantic segmentation information | |
FaceX | AAAI 2019 | ✔️ | ❌ | Labeled facial sketches | |
Creative Sketch | ICLR 2021 | ✔️ | ❌ | With annotated part segmentation | |
ImageNet-Sketch | NeurIPS 2019 | ❌ | ❌ | 50 images for each of the 1000 ImageNet classes | |
Scene-level | SketchyScene | ECCV 2018 | ❌ | ✔️ | With semantic/instance segmentation information |
CMPlaces | TPAMI 2018 | ❌ | ✔️ | Cross-modal scene dataset | |
Context-Skecth | Expressive 2018 | ❌ | ✔️ | Context-based scene sketches for co-classification | |
SketchyCOCO | CVPR 2020 | ❌ | ✔️ | Scene sketch, segmentation and normal images | |
Drawing from photos | Photo-Sketching | WACV 2019 | ✔️ | ✔️ | ScenePhoto-sketch paired |
Tracing-vs-Freehand | SIGGRAPH 2021 | ✔️ | ✔️ | Tracings and freehand drawings of images | |
Rough sketch | Da Vinci | CGI 2018 | ❌ | ✔️ | Line drawing restoration dataset |
Rough Sketch Benchmark | SIGGRAPH Asia 2020 | ✔️ | ✔️ | Rough and clean sketch pairs (only for evaluation) | |
CAD | OpenSketch | SIGGRAPH Asia 2019 | ✔️ | ❌ | Product Design Sketches |
SketchGraphs | ICML 2020 Workshop | ✔️ | ❌ | Sketches extracted from real-world CAD models | |
Fusion 360 Gallery | SIGGRAPH 2021 | ✔️ | ❌ | For 'sketch and extrude' designs | |
FloorPlanCAD | ICCV 2021 | ✔️ | ❌ | With instance and semantic annotations |
Level | Paper | Source | Code/Project Link |
Instance-level | A Neural Representation of Sketch Drawings (sketch-rnn) | ICLR 2018 | [Code] [Project] [Demo] |
Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories | [Code] | ||
AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches | AAAI 2019 | [Project] | |
Stroke-based sketched symbol reconstruction and segmentation (stroke-rnn) | |||
BézierSketch: A generative model for scalable vector sketches | ECCV 2020 | ||
Pixelor: A Competitive Sketching AI Agent. So you think you can beat me? | SIGGRAPH Asia 2020 | [Project] [Code] | |
Creative Sketch Generation | ICLR 2021 | [Project] [Code] | |
Computer-Aided Design as Language | arxiv 2105 |
- vector image generation
Data type | Paper | Source | Code/Project Link |
Facial | Style and abstraction in portrait sketching | TOG 2013 | |
Making Robots Draw A Vivid Portrait In Two Minutes | IROS 2020 | [Code] [Project] | |
RoboCoDraw: Robotic Avatar Drawing with GAN-based Style Transfer and Time-efficient Path Optimization | AAAI 2020 | [Code] | |
Instance-level | Free-Hand Sketch Synthesis with Deformable Stroke Models | IJCV 2017 | [Project] [code] |
Learning to Sketch with Shortcut Cycle Consistency | CVPR 2018 | [Code1] [Code2] | |
Learning Deep Sketch Abstraction | CVPR 2018 | ||
Technical Drawings | Deep Vectorization of Technical Drawings | ECCV 2020 | [Project] [code] |
Scene-level | Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale | AAAI 2021 | [Code] |
- pixelwise image generation
Level | Paper | Source | Code/Project Link |
Facial | APDrawingGAN: Generating Artistic Portrait Drawings from Face Photos with Hierarchical GANs | CVPR 2019 | [Code] [Demo] |
Instance-level | Deep Factorised Inverse-Sketching | ECCV 2018 | |
Making better use of edges for sketch generation | JEI 2018 | ||
Synthesizing human-like sketches from natural images using a conditional convolutional decoder | WACV 2020 | [Code] | |
Scene-level | Photo-Sketching: Inferring Contour Drawings from Images | WACV 2019 | [Code] [Project] |
Level | Paper | Source | Code/Project Link |
---|---|---|---|
Facial | Text2Sketch: Learning Face Sketch from Facial Attribute Text | ICIP 2018 | |
Scene-level | Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications | UIST 2019 | |
Scene-level | Scones: Towards Conversational Authoring of Sketches | IUI 2020 | |
Arbitrary | CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders | arxiv 2106 | [code] |
Arbitrary | Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts | arxiv 2109 | [code] [project] |
Paper | Source | Code/Project Link |
---|---|---|
DeepShapeSketch : Generating hand drawing sketches from 3D objects | IJCNN 2019 | |
Neural Contours: Learning to Draw Lines from 3D Shapes | CVPR 2020 | [project] [code] |
Cloud2Curve: Generation and Vectorization of Parametric Sketches | CVPR 2021 | [project] |
Neural Strokes: Stylized Line Drawing of 3D Shapes | ICCV 2021 | [code] |
This means translating a disordered vector sketch into an ordered sequential sketch imitating human's drawing order. The appearance of the sequential sketch is exactly the same as the pixelwise one.
Paper | Source | Code/Project Link |
---|---|---|
Animated Construction of Line Drawings | SIGGRAPH ASIA 2011 | [Project] [code] [Demo] |
Here we list sketch synthesis based on other image types, like Manga, line art, rough sketch, etc.
- Hand drawn line art / rough sketch (a.k.a. Vectorization / Sketch Simplification)
a) Datasets and benchmark
Paper | Source | Code/Project Link |
---|---|---|
A Benchmark for Rough Sketch Cleanup | SIGGRAPH Asia 2020 | [Project] [Code] |
b) Optimization-based approaches
c) Learning-based approaches
- Manga (Comics)
Paper | Source | Input Type | Output Type | Code/Project Link |
---|---|---|---|---|
Deep extraction of manga structural lines | SIGGRAPH 2017 | raster | raster | [Code] |
Manga Filling Style Conversion with Screentone Variational Autoencoder | SIGGRAPH Asia 2020 | raster | raster | [Project] [Code] |
Generating Manga from Illustrations via Mimicking Manga Workflow | CVPR 2021 | raster | raster | [Project] [Code] |
Vectorization of Raster Manga by Deep Reinforcement Learning | arxiv 2110 | raster | vector |
Here we focus on learning-based vector graphics generation without depending on vector training data, and traditional vectorization algorithms.
- Learning with external black-box (non-differentiable) rendering simulator
Paper | Source | Code/Project Link |
---|---|---|
Synthesizing Programs for Images using Reinforced Adversarial Learning | ICML 2018 | [Code] |
Unsupervised Doodling and Painting with Improved SPIRAL | arxiv 1910 | [Project] |
- Learning with built-in differentiable rendering module
- Vectorization
Paper | Source | Code/Project Link |
---|---|---|
Depixelizing pixel art | SIGGRAPH 2011 | |
Perception-Driven Semi-Structured Boundary Vectorization | SIGGRAPH 2018 | [Webpage] |
PolyFit: Perception-aligned Vectorization of Raster Clip-art via Intermediate Polygonal Fitting | SIGGRAPH 2020 | [Webpage] [Code] |
ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis | TVCG 2021 |