GenerativeMathematical
◆ The PatternGenerating by learning to reverse a noise process (DDPM, Stable Diffusion)
Diffusion models learn to denoise data. The forward process adds noise; a neural network learns to reverse it.
Forward: q(xₜ|xₜ₋₁) = N(xₜ; √(1−β)·xₜ₋₁, β·I)
Adds noise β at each step. After T steps, data ≈ pure Gaussian noise.
xₜ = √ᾱₜ·x₀ + √(1−ᾱₜ)·ε ε~N(0,I)
Closed form for any timestep t.
L = E[||ε − εθ(xₜ,t)||²]
Training: predict the noise ε that was added.
// Forward diffusion — noise being added over timesteps
Timestep t0
DDPM → DDIM → Latent Diffusion: DDPM (2020). DDIM made sampling 10–50× faster. Latent Diffusion (Stable Diffusion) runs in compressed latent space — enabling image generation on consumer GPUs.
Pattern bridge: Adding noise then learning to reverse it mirrors the central limit theorem in reverse — from Gaussian noise back to structured signal. In markets, euphoria and panic inject noise that mean-reverts to equilibrium.
Worked example
DDPM’s schedule adds noise over 1,000 steps. The share of the original signal left is 0.9999 after one step, 0.947 after 100, 0.280 after 500 and 0.006 after 1,000 — essentially pure noise. The model learns to undo one small step at a time, and generation runs the whole chain backwards.
Where it misleads
- The linear schedule destroys information quickly in the middle; a cosine schedule spreads it more evenly (Nichol & Dhariwal 2021).
- A thousand denoising steps make sampling slow; faster samplers trade steps for quality.
- Likelihood and sample quality do not always improve together.
In code
T = 1000
beta = np.linspace(1e-4, 0.02, T) # DDPM's linear noise schedule
alpha_bar = np.cumprod(1 - beta)
signal = {t: float(np.sqrt(alpha_bar[t - 1]).round(4)) for t in (1, 100, 500, 1000)} # share of the original leftAssumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.
Sources
- J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan & S. Ganguli, “Deep Unsupervised Learning Using Nonequilibrium Thermodynamics”, ICML, 2015
- J. Ho, A. Jain & P. Abbeel, “Denoising Diffusion Probabilistic Models”, NeurIPS, 2020
- A. Q. Nichol & P. Dhariwal, “Improved Denoising Diffusion Probabilistic Models”, ICML, 2021
Last reviewed October 2026 · How this site checks its content · Report a mistake
Linked from 1