jiangy2 / diffusion-models-papers-survey-taxonomy

Diffusion model papers, survey, and taxonomy

Diffusion Models: A Comprehensive Survey of Methods and Applications

This repo is constructed for collecting and categorizing papers about diffusion models according to our survey paper——Diffusion Models: A Comprehensive Survey of Methods and Applications

Overview

Catalogue

Algorithm Taxonomy

Sampling-Acceleration Enhancement

• Discretization Optimization
• Non-Markovian Process
• Partial Sampling

Likelihood-Maximization Enhancement

• Noise Schedule Optimization
• Learnable Reverse Variance
• Objectives Designing

Data-Generalization Enhancement

• Feature Space Unification
• Data-Dependent Transition Kernels

Application Taxonomy

• Computer Vision
• Natural Language Processing
• Multi-Modal Learning
• Molecular Graph Modeling
• Time-Series Modeling
• Adversarial Purification
• Waveform Signal Processing

Connections with Other Generative Models

• Variational Autoencoder
• Generative Adversarial Network
• Normalizing Flow
• Autoregressive Models
• Energy-Based Models

Algorithm Taxonomy

1. Sampling-Acceleration Enhancement

1.1. Discretization Optimization

Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction

Diffusion Schrödinger bridge with applications to score-based generative modeling

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

Gotta Go Fast When Generating Data with Score-Based Models

Elucidating the Design Space of Diffusion-Based Generative Models

Pseudo Numerical Methods for Diffusion Models on Manifolds

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Step

Score-Based Generative Modeling through Stochastic Differential Equations

Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality

Fast Sampling of Diffusion Models with Exponential Integrator

1.2. Non-Markovian Process

Denoising Diffusion Implicit Models

Pseudo Numerical Methods for Diffusion Models on Manifolds

gDDIM: Generalized denoising diffusion implicit models

Learning fast samplers for diffusion models by differentiating through sample quality

1.3. Partial Sampling

Progressive Distillation for Fast Sampling of Diffusion Models

Accelerating Diffusion Models via Early Stop of the Diffusion Process

Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed

Truncated Diffusion Probabilistic Models

2. Likelihood-Maximization Enhancement

2.1. Noise Schedule Optimization

Variational diffusion models

Improved denoising diffusion probabilistic models

2.2. Learnable Reverse Variance

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Improved denoising diffusion probabilistic models

2.3. Objectives Designing

Maximum likelihood training of score-based diffusion models

Maximum Likelihood Training for Score-based Diffusion ODEs by High Order Denoising Score Matching

A variational perspective on diffusion-based generative models and score matching

Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

3. Data-Generalization Enhancement

3.1. Feature Space Unification

Pseudo Numerical Methods for Diffusion Models on Manifolds

Score-based generative modeling in latent space

Riemannian Score-Based Generative Modeling

Diffusion priors in variational autoencoders

3.2. Data-Dependent Transition Kernels

GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation

Permutation invariant graph generation via score-based generative modeling

Vector quantized diffusion model for text-to-image synthesis

Structured Denoising Diffusion Models in Discrete State-Spaces

Vector Quantized Diffusion Model with CodeUnet for Text-to-Sign Pose Sequences Generation

Application Taxonomy

1. Computer Vision

• Image Super Resolution and Inpainting

  • SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models

  • Image Super-Resolution via Iterative Refinement

  • High-Resolution Image Synthesis with Latent Diffusion Models

  • Repaint: Inpainting using denoising diffusion probabilistic models.

  • Palette: Image-to-image diffusion models.

  • Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models

  • Cascaded Diffusion Models for High Fidelity Image Generation.

  • Conditional image generation with score-based diffusion models

  • Unsupervised Medical Image Translation with Adversarial Diffusion Models

  • Score-based diffusion models for accelerated MRI

  • Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

  • MR Image Denoising and Super-Resolution Using Regularized Reverse Diffusion

• Semantic Segmentation

  • Label-Efficient Semantic Segmentation with Diffusion Models.
  • Decoder Denoising Pretraining for Semantic Segmentation.

• Anomaly Detection

  • AnoDDPM: Anomaly Detection With Denoising Diffusion Probabilistic Models Using Simplex Noise
  • Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models.

• Point Cloud Completion and Generation

  • 3d shape generation and completion through point-voxel diffusion
  • Diffusion probabilistic models for 3d point cloud generation
  • A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
  • Let us Build Bridges: Understanding and Extending Diffusion Generative Models.

• Video Generation

  • Flexible Diffusion Modeling of Long Videos
  • Video diffusion models
  • Diffusion probabilistic modeling for video generation
  • MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model.

2. Natural Language Processing

• Structured denoising diffusion models in discrete state-spaces
• Diffusion-LM Improves Controllable Text Generation.
• Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning

3. Multi-Modal Learning

• Text-to-Image Generation
  • Blended diffusion for text-driven editing of natural images
  • Hierarchical Text-Conditional Image Generation with CLIP Latents
  • Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
  • GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
  • Vector quantized diffusion model for text-to-image synthesis.
  • Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
• Text-to-Audio Generation
  • Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
  • Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data
  • Diffsound: Discrete Diffusion Model for Text-to-sound Generation
  • ItôTTS and ItôWave: Linear Stochastic Differential Equation Is All You Need For Audio Generation
  • Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models
  • EdiTTS: Score-based Editing for Controllable Text-to-Speech.
  • ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech.

4. Molecular Graph Modeling

• Torsional Diffusion for Molecular Conformer Generation.
• Equivariant Diffusion for Molecule Generation in 3D
• Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models
• GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation
• Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem
• Diffusion-based Molecule Generation with Informative Prior Bridge
• Learning gradient fields for molecular conformation generation
• Predicting molecular conformation via dynamic graph score matching.

5. Time-Series Modeling

• Time Series Imputation
  • CSDI: Conditional score-based diffusion models for probabilistic time series imputation
  • Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models
  • Neural Markov Controlled SDE: Stochastic Optimization for Continuous-Time Data
• Time Series Forecasting
  • Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting
  • Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models

6. Adversarial Purification

• Diffusion Models for Adversarial Purification
• Adversarial purification with score-based generative models
• Threat Model-Agnostic Adversarial Defense using Diffusion Models
• Guided Diffusion Model for Adversarial Purification
• Guided Diffusion Model for Adversarial Purification from Random Noise
• PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognition.

7. Waveform Signal Processing

• WaveGrad: Estimating Gradients for Waveform Generation.
• DiffWave: A Versatile Diffusion Model for Audio Synthesis

Connections with Other Generative Models

1. Variational Autoencoder

• Understanding Diffusion Models: A Unified Perspective
• A variational perspective on diffusion-based generative models and score matching
• Score-based generative modeling in latent space

2. Generative Adversarial Network

• Diffusion-GAN: Training GANs with Diffusion.
• Tackling the generative learning trilemma with denoising diffusion gans

3. Normalizing Flow

• Diffusion Normalizing Flow
• Interpreting diffusion score matching using normalizing flow

4. Autoregressive Models

• Autoregressive Diffusion Models.
• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting.

5. Energy-Based Models

• Learning Energy-Based Models by Diffusion Recovery Likelihood
• Latent Diffusion Energy-Based Model for Interpretable Text Modeling

Citing

If you find this work useful, please cite our paper:

@article{yang2022diffusion,
  title={Diffusion Models: A Comprehensive Survey of Methods and Applications},
  author={Yang, Ling and Zhang, Zhilong and Hong, Shenda},
  journal={arXiv preprint arXiv:2209.00796},
  year={2022}
}