9/ Check it yourself:

🔗: github.com/khaledkah/tv...
📄: www.arxiv.org/abs/2502.08598

Thanks to Khaled, Winnie, Oliver, Klaus, and Shin for the cool collaboration as well as @bifold.berlin, TU Berlin, RIKEN, and DeepMind

8/ Takeaway: Exploding TV isn’t needed. Control TV + SNR separately for faster, better sampling. Method generalizes across domains (molecules, images).

7/ Why it works? Our empirical analysis shows:

1. Straight trajectories near data (t ≈ 0) are important (see in the inset plot)
2. Broad support of pₜ(𝐱) early on → robust to errors (note how SMLD goes from small to huge range instead of staying the same)

6/ Images: Matches EDM with uniform grid

No fancy time grids like in EDM needed! VP-ISSNR on CIFAR-10/FFHQ ≈ EDM but with fewer hyperparameters!

5/ Molecules in 8 Steps:

VP-ISSNR achieves 74% stability with 8 steps, 95% with 64 (SDE). Beats all baselines!

4/ We propose a new VP schedule 📈:

Exponential inverse sigmoid SNR (ISSNR)→ rapid decay at start/end. Generalizes Optimal Transport Flow Matching.

3/ VP variants improve existing schedules:

Take SMLD/EDM (exploding TV) → force TV=1. Result: +30% stability for molecules with 8 steps

(x-axis is NFE=number of function evals).

2/ Most schedules (like EDM by Karras or SMLD (Song & Ermon) let TV explode (VE=variance exploding).

We show constant TV (variance preserving, VP) + optimized SNR works better (ISSNR)!

(it's a wild table, sorry, but notice our VP variants I circled)

1/ Problem: Diffusion models are slow due to repeated evals but reducing steps hurts quality if the noise schedule isn’t optimal. Other schedules passively adjust variance. Can we do better?

🔑Insight: control Total Variance (TV) and signal-to-noise-ratio (SNR) independently!

acab includes the raclette police 💡

i guess their claim would be that it blows up for mysterious NN reasons rather than integrator or time step. 2 fs is a bit of chonky step, i agree, but if it explodes at say 0.1 fs i'd start wondering about the NN more than about the time step

stuff like this, (Ala)_2 at 2 fs or water at 1 fs? at <=0.5 fs they wouldn't explode for curl free forces, i assume?

why was that paper bad? i thought it was more of a benchmark than proposing their own thing anyways?

i raise you to ful midammis oml

Seems like it refers to DFT. They actually give an intuition from metallurgy in the appendix about simulated annealing. I didn't know it's about removing impurities 🙉

Excellent question. Alas, I can't say i like mcmc (or tmcmc?) as a word either. A string of non-descript names is just ... Eugh

Ah yes, 'annealing', i do it every day and have a super intuitive understanding of what it is. In fact, im annealing right now.