Awesome-Architectural-Graphics

A collection of papers about computer graphics research projects in the last few years that involved potential architectural applications. Mainly focus on architectural geometry, vector fields, shape analysis, inverse CAD design, etc.

Feel free to create a PR or an issue.

Outlines

• 1. Design Synthesis
  • 1) 2D Floorplan synthesis
  • 2) 3D Shape generation
• 2. (Inverse) CAD Modeling
  • 1) Sketch-to-CAD
  • 2) Photo-to-CAD
  • 3) Text/Attribute-to-CAD
  • 4) Procedural Modeling
• 3. Geometry Optimization(Topology Optimization Excluded)
  • 1) Fabrication-oriented Geometry
  • 2) Structure-oriented Geometry
  • 3) Deployment-oriented Geometry
  • 4) Curve/Ribbon/Crease
• 4. Directional Field
• 5. Design Space

---

1. Creative ideas & design

Here Vector strokes means having svg data. With photos means having the photo-sketch paired data.

Level	Dataset	Source	Vector strokes	With photos	Notes
Characters	Omniglot		✔️	❌	Alphabets characters
	KanjiVG		✔️	❌	Chinese characters
	Kuzushiji		❌	❌	Japanese characters
Instance-level	TU-Berlin	SIGGRAPH 2012	✔️	❌	Multi-category hand sketches
	Sketchy	SIGGRAPH 2016	✔️	✔️	Multi-category photo-sketch paired
	QuickDraw	ICLR 2018	✔️	❌	Multi-category hand sketches
	QMUL-Shoe-Chair-V2	CVPR 2016	✔️	✔️	Only two categories
	Sketch Perceptual Grouping (SPG)	ECCV 2018	✔️	❌	With part-level semantic segmentation information
	FaceX	AAAI 2019	✔️	❌	Labeled facial sketches
	Creative Sketch	ICLR 2021	✔️	❌	With annotated part segmentation
	ImageNet-Sketch	NeurIPS 2019	❌	❌	50 images for each of the 1000 ImageNet classes
Scene-level	SketchyScene	ECCV 2018	❌	✔️	With semantic/instance segmentation information
	CMPlaces	TPAMI 2018	❌	✔️	Cross-modal scene dataset
	Context-Skecth	Expressive 2018	❌	✔️	Context-based scene sketches for co-classification
	SketchyCOCO	CVPR 2020	❌	✔️	Scene sketch, segmentation and normal images
	FS-COCO	ECCV 2022	✔️	✔️	Scene sketches with text description
Drawing from photos	Photo-Sketching	WACV 2019	✔️	✔️	ScenePhoto-sketch paired
	Tracing-vs-Freehand	SIGGRAPH 2021	✔️	✔️	Tracings and freehand drawings of images
Portrait	APDrawing	CVPR 2019	❌	✔️	Portrait-sketch paired
Rough sketch	Da Vinci	CGI 2018	❌	✔️	Line drawing restoration dataset
	Rough Sketch Benchmark	SIGGRAPH Asia 2020	✔️	✔️	Rough and clean sketch pairs (only for evaluation)
CAD	ld3d	SIGGRAPH 2008	❌	❌	Line Drawings of 3D Shapes
	OpenSketch	SIGGRAPH Asia 2019	✔️	❌	Product Design Sketches
	SketchGraphs	ICML 2020 Workshop	✔️	❌	Sketches extracted from real-world CAD models
	Fusion 360 Gallery	SIGGRAPH 2021	✔️	❌	For 'sketch and extrude' designs
	FloorPlanCAD	ICCV 2021	✔️	❌	With instance and semantic annotations

2. Architectural Geometry

1) Curve network

Level	Paper	Source	Code/Project Link
Curve-study	Stability-Aware Simplification of Curve Networks	SIGGRAPH 2022	[Code] [Project] [Presentation]
	3D Weaving with Curved Ribbons	SIGGRAPH 2021	[Project] [Web Demo]
	THE DESIGN SPACE OF PLANE ELASTIC CURVES	SIGGRAPH 2021	[Project] [Code]
	Stroke-based sketched symbol reconstruction and segmentation (stroke-rnn)
	BézierSketch: A generative model for scalable vector sketches	ECCV 2020
	Pixelor: A Competitive Sketching AI Agent. So you think you can beat me?	SIGGRAPH Asia 2020	[Project] [Code]
	Creative Sketch Generation	ICLR 2021	[Project] [Code]
	Computer-Aided Design as Language	arxiv 2105

---

2) Discrete nets/meshes

• vector image generation

Data type	Paper	Source	Code/Project Link
Facial	Style and abstraction in portrait sketching	TOG 2013
	Making Robots Draw A Vivid Portrait In Two Minutes	IROS 2020	[Code] [Project]
	RoboCoDraw: Robotic Avatar Drawing with GAN-based Style Transfer and Time-efficient Path Optimization	AAAI 2020	[Code]
Instance-level	Free-Hand Sketch Synthesis with Deformable Stroke Models	IJCV 2017	[Project] [code]
	Learning to Sketch with Shortcut Cycle Consistency	CVPR 2018	[Code1] [Code2]
	Learning Deep Sketch Abstraction	CVPR 2018
	CLIPasso: Semantically-Aware Object Sketching	SIGGRAPH 2022	[Project] [Code]
Technical Drawings	Deep Vectorization of Technical Drawings	ECCV 2020	[Project] [code]
Scene-level	Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale	AAAI 2021	[Code]

• pixelwise image generation

Level	Paper	Source	Code/Project Link
Facial	ArtLine	Online demo	[Code]
	APDrawingGAN: Generating Artistic Portrait Drawings from Face Photos with Hierarchical GANs	CVPR 2019	[Code] [Demo]
	Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping	CVPR 2020	[Code]
	Line Drawings for Face Portraits From Photos Using Global and Local Structure Based GANs	TPAMI 2020	[Code]
	Quality Metric Guided Portrait Line Drawing Generation from Unpaired Training Data	TPAMI 2022	[Code]
Instance-level	Deep Factorised Inverse-Sketching	ECCV 2018
	Making better use of edges for sketch generation	JEI 2018
	Synthesizing human-like sketches from natural images using a conditional convolutional decoder	WACV 2020	[Code]
Scene-level	Photo-Sketching: Inferring Contour Drawings from Images	WACV 2019	[Code] [Project]
	Learning to generate line drawings that convey geometry and semantics	CVPR 2022	[Code] [Project]

---

3) Deployable system

Level	Paper	Source	Code/Project Link
Facial	Text2Sketch: Learning Face Sketch from Facial Attribute Text	ICIP 2018
Scene-level	Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications	UIST 2019
Scene-level	Scones: Towards Conversational Authoring of Sketches	IUI 2020
Arbitrary	CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders	arxiv 2106	[code]
Arbitrary	Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts	arxiv 2109	[code] [project]

---

4) 3D shape-to-sketch

Paper	Source	Code/Project Link
DeepShapeSketch : Generating hand drawing sketches from 3D objects	IJCNN 2019
Neural Contours: Learning to Draw Lines from 3D Shapes	CVPR 2020	[project] [code]
Cloud2Curve: Generation and Vectorization of Parametric Sketches	CVPR 2021	[project]
Neural Strokes: Stylized Line Drawing of 3D Shapes	ICCV 2021	[code]

---

5) Sketch(vector)-to-sketch(vector)

This means translating a disordered vector sketch into an ordered sequential sketch imitating human's drawing order. The appearance of the sequential sketch is exactly the same as the pixelwise one.

Paper	Source	Code/Project Link
Animated Construction of Line Drawings	SIGGRAPH ASIA 2011	[Project] [code] [Demo]

---

6) Art-to-sketch

Here we list sketch synthesis based on other image types, like Manga, line art, rough sketch, etc.

• Line art

a) Datasets and benchmark

Paper	Source	Code/Project Link
A Benchmark for Rough Sketch Cleanup	SIGGRAPH Asia 2020	[Project] [Code]

b) Vector-to-Vector

Paper	Source	Code/Project Link	*-based
Closure-aware Sketch Simplification	SIGGRAPH Asia 2015	[Project]	Optimization
StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings	SIGGRAPH 2018	[Project]	Optimization
StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters	SIGGRAPH 2021	[Project] [code]	Optimization

c) Raster-to-Vector (a.k.a. Vectorization)

Paper	Source	Code/Project Link	*-based
Topology-Driven Vectorization of Clean Line Drawings	TOG 2013		Optimization
Fidelity vs. Simplicity: a Global Approach to Line Drawing Vectorization	SIGGRAPH 2016	[Project]	Optimization
A Delaunay triangulation based approach for cleaning rough sketches	C&G 2018	[Code]	Optimization
Semantic Segmentation for Line Drawing Vectorization Using Neural Networks	EG 2018	[project] [code]	Learning
Deep Line Drawing Vectorization via Line Subdivision and Topology Reconstruction	PG 2019		Learning
Inertia-based Fast Vectorization of Line Drawings	PG 2019		Optimization
Vectorization of Line Drawings via Polyvector Fields	TOG 2019	[Code]	Optimization
Integer-Grid Sketch Simplification and Vectorization	SGP 2020	[Project] [Code]	Optimization
Deep Vectorization of Technical Drawings	ECCV 2020	[project] [code]	Learning
General Virtual Sketching Framework for Vector Line Art	SIGGRAPH 2021	[project] [code]	Learning
Keypoint-Driven Line Drawing Vectorization via PolyVector Flow	SIGGRAPH Asia 2021	[project]	Learning + Optimization
End-to-end Line Drawing Vectorization	AAAI 2022		Learning
Vectorizing Line Drawings of Arbitrary Thickness via Boundary-based Topology Reconstruction	CGF 2022		Heuristic

• Rough sketch simplification

Paper	Source	Code/Project Link
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup	SIGGRAPH 2016	[Code] [Project]
Mastering Sketching: Adversarial Augmentation for Structured Prediction	SIGGRAPH 2018	[Code] [Project]
Real-Time Data-Driven Interactive Rough Sketch Inking	SIGGRAPH 2018	[Code] [Project]
Perceptual-aware Sketch Simplification Based on Integrated VGG Layers	TVCG 2019

• Manga (Comics)

Paper	Source	Code/Project Link
Deep extraction of manga structural lines	SIGGRAPH 2017	[Code]
Manga Filling Style Conversion with Screentone Variational Autoencoder	SIGGRAPH Asia 2020	[Project] [Code]
Generating Manga from Illustrations via Mimicking Manga Workflow	CVPR 2021	[Project] [Code]
MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing	AAAI 2021
Vectorization of Raster Manga by Deep Reinforcement Learning	arxiv 2110

3. Inverse CAD Modeling

Here we focus on learning-based vector graphics generation without depending on vector training data, and traditional vectorization algorithms.

• Learning with external black-box (non-differentiable) rendering simulator

Paper	Source	Code/Project Link
Synthesizing Programs for Images using Reinforced Adversarial Learning	ICML 2018	[Code]
Unsupervised Doodling and Painting with Improved SPIRAL	arxiv 1910	[Project]

• Learning with built-in differentiable rendering module

Paper	Source	Code/Project Link
Towards Layer-wise Image Vectorization	CVPR 2022	[code] [project]
Paint Transformer: Feed Forward Neural Painting with Stroke Prediction	ICCV 2021	[code]
Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes	CVPR 2021	[code]
Im2Vec: Synthesizing Vector Graphics without Vector Supervision	CVPR 2021	[Project] [code]
Stylized Neural Painting	CVPR 2021	[Code] [project]
Learning to Paint With Model-based Deep Reinforcement Learning	ICCV 2019	[code]
Strokenet: A neural painting environment	ICLR 2019	[Code]
Neural Painters: A learned differentiable constraint for generating brushstroke paintings	arxiv 1904	[Code]
Learning to Sketch with Deep Q Networks and Demonstrated Strokes	arxiv 1810
Unsupervised Image to Sequence Translation with Canvas-Drawer Networks	arxiv 1809

• Vectorization

Paper	Source	Code/Project Link
Depixelizing pixel art	SIGGRAPH 2011
Perception-Driven Semi-Structured Boundary Vectorization	SIGGRAPH 2018	[Webpage]
PolyFit: Perception-aligned Vectorization of Raster Clip-art via Intermediate Polygonal Fitting	SIGGRAPH 2020	[Webpage] [Code]
ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis	TVCG 2021
TCB-Spline-Based Image Vectorization	TOG 2022