peterzs / designing-perceptual-puzzles-by-differentiating-probabilistic-programs

Supplementary materials for our SIGGRAPH 2022 paper

Designing Perceptual Puzzles by Differentiating Probabilistic Programs

This repository contains source code to accompany the SIGGRAPH paper Designing Perceptual Puzzles by Differentiating Probabilistic Programs (Chandra, Li, Tenenbaum, and Ragan-Kelley 2022).

We design new visual illusions by finding "adversarial examples" for principled models of human perception — specifically, for probabilistic models, which treat vision as Bayesian inference. To perform this search efficiently, we design a differentiable probabilistic programming language, whose API exposes MCMC inference as a first-class differentiable function. We demonstrate our method by automatically creating illusions for three features of human vision: color constancy, size constancy, and face perception.

@InProceedings{chandra2022designing,
  title = {Designing Perceptual Puzzles by Differentiating Probabilistic Programs},
  author = {Kartik Chandra and Tzu-Mao Li and Joshua Tenenbaum and Jonathan Ragan-Kelley},
  booktitle = {Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings (SIGGRAPH '22 Conference Proceedings)},
  month = {aug},
  year = {2022},
  doi = {10.1145/3528233.3530715}
}

---

Contents

• Differentiable Probabilistic Programming (Sec 2)
  • razor.py: our differentiable PPL's implementation (Sec 2.2)
  • reversible.py: implementation of reversible learning in JAX (Sec 2.3)
  • Thermometer.ipynb: concrete implementation of our worked example (Sec 3)
• Applications (Sec 3)
  • Color constancy.ipynb (Sec 4.1)
    • cc/: pre-rendered masks to be composited by renderer
  • Size constancy.ipynb (Sec 4.2)
  • Face perception.ipynb (Sec 4.3)
    • softras.py: implementation of SoftRas differentiable renderer in JAX
    • basel_cmplx.npz: low-poly simplified mesh of the Basel Face Model

Requirements:

• Python 3 (3.9.5), ImageMagick
• jax (0.2.20), jaxlib (0.1.71+cuda11)
• Jupyter Lab, numpy, matplotlib, imageio, tqdm