Phew... that's one less thing on my list of things that I don't know about (infty - 1)
February 18, 2024 at 8:13 PM
Phew... that's one less thing on my list of things that I don't know about (infty - 1)
"ES methods convolve the loss surface with a Gaussian distribution in the learnable parameter space, making it smoother and infinitely differentiable" - this very much reminds me of the score-matching interpretation of diffusion models. Taking a look over breakfast...
February 16, 2024 at 8:01 AM
"ES methods convolve the loss surface with a Gaussian distribution in the learnable parameter space, making it smoother and infinitely differentiable" - this very much reminds me of the score-matching interpretation of diffusion models. Taking a look over breakfast...
Very cool! One thing I'm still not sure about is relevance to meta-learning (Discussion E). It's hard to guess from the looking at color maps, but although "best hparams are near the edge", is a perilous journey into fractal-land warranted for the reward vs good hparams in a more stable region?
February 14, 2024 at 11:34 AM
Very cool! One thing I'm still not sure about is relevance to meta-learning (Discussion E). It's hard to guess from the looking at color maps, but although "best hparams are near the edge", is a perilous journey into fractal-land warranted for the reward vs good hparams in a more stable region?