o1-preview beat my family zero-shot at Codenames when paired with the weakest guesser. Clues weren’t amazing when picking word pairs for itself, but very impressive ability to relate words when given a collection and asked for a clue that relates them all.

jax.config.update(“jax_explain_cache_misses”, True) is an extremely useful tool for figuring out what parts of your Jax program are getting recompiled and why.

A research idea that I would love to see someone take further is “reverse” positional encodings that count down to an EOS token. The training setup is easy and it could offer an interesting way to bias the length+style of LM responses at runtime. Want ~100 token response? Count down from 100.

Had a great week at SC24! Very exciting to see the ways that diffusion and other ML algorithms are transforming scientific computing. The tooling community also seems to be making big strides to make HPC more accessible to a broader scientific community. Look forward to next year!