Keyon Vafa
@keyonv.bsky.social
Postdoctoral fellow at Harvard Data Science Initiative | Former computer science PhD at Columbia University | ML + NLP + social sciences
https://keyonvafa.com
https://keyonvafa.com
We apply these probes to orbital, lattice, and Othello problems.
Starting with orbits: we encode solar systems as sequences and train a transformer on 10M solar systems (20B tokens)
The model makes accurate predictions many timesteps ahead. Predictions for our solar system:
Starting with orbits: we encode solar systems as sequences and train a transformer on 10M solar systems (20B tokens)
The model makes accurate predictions many timesteps ahead. Predictions for our solar system:
July 14, 2025 at 1:50 PM
We apply these probes to orbital, lattice, and Othello problems.
Starting with orbits: we encode solar systems as sequences and train a transformer on 10M solar systems (20B tokens)
The model makes accurate predictions many timesteps ahead. Predictions for our solar system:
Starting with orbits: we encode solar systems as sequences and train a transformer on 10M solar systems (20B tokens)
The model makes accurate predictions many timesteps ahead. Predictions for our solar system:
Can an AI model predict perfectly and still have a terrible world model?
What would that even mean?
Our new ICML paper (poster tomorrow!) formalizes these questions.
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
What would that even mean?
Our new ICML paper (poster tomorrow!) formalizes these questions.
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
July 14, 2025 at 1:50 PM
Can an AI model predict perfectly and still have a terrible world model?
What would that even mean?
Our new ICML paper (poster tomorrow!) formalizes these questions.
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
What would that even mean?
Our new ICML paper (poster tomorrow!) formalizes these questions.
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵