Happy little accidents...

Second, sampling from the distribution with a Langevin Dynamics simulation. The algorithm is almost identical to gradient descent with momentum, but we add just the right amount of noise to the momentum at each step.

Comparison of optimization and sampling from a distribution defined by an energy function. I use a continuous version of the Ising model spin lattice energy.

First, optimization from a random initial state using gradient descent with momentum, using the SGD optimizer in PyTorch.