AI & Junk Food

When you optimize food for short-term preference, you get sugary, salty brain hacks. Yum, unhealthy in excess.

When you optimize models for short-term preference, you get flattering, verbose, sloppy brain hacks. Cool, unhealthy in excess.

Do you agree? What's 'healthy food' for AI?

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish to those for which training diverges.

Even better, a video: vimeo.com/903855670

Hello bsky, I'm not sure what I'll post where - Twitter/X is still my default. But I do also put most of the interesting stuff on my site's "Everything" page, which has an RSS feed, if you want to see that stuff without X :)
johnowhitaker.dev/all.html