Pete Shaw
@ptshaw.bsky.social
Research Scientist at Google DeepMind. Mostly work on ML, NLP, and BioML. Based in Seattle.
http://ptshaw.com
http://ptshaw.com
Excited to share a new paper that aims to narrow the conceptual gap between the idealized notion of Kolmogorov complexity and practical complexity measures for neural networks.
October 1, 2025 at 2:11 PM
Excited to share a new paper that aims to narrow the conceptual gap between the idealized notion of Kolmogorov complexity and practical complexity measures for neural networks.
Reposted by Pete Shaw
Excited to share 𝐈𝐧𝐟𝐀𝐥𝐢𝐠𝐧!
Alignment optimization objective implicitly assumes 𝘴𝘢𝘮𝘱𝘭𝘪𝘯𝘨 from the resulting aligned model. But we are increasingly using different and sometimes sophisticated inference-time compute algorithms.
How to resolve this discrepancy?🧵
Alignment optimization objective implicitly assumes 𝘴𝘢𝘮𝘱𝘭𝘪𝘯𝘨 from the resulting aligned model. But we are increasingly using different and sometimes sophisticated inference-time compute algorithms.
How to resolve this discrepancy?🧵
January 1, 2025 at 7:59 PM
Excited to share 𝐈𝐧𝐟𝐀𝐥𝐢𝐠𝐧!
Alignment optimization objective implicitly assumes 𝘴𝘢𝘮𝘱𝘭𝘪𝘯𝘨 from the resulting aligned model. But we are increasingly using different and sometimes sophisticated inference-time compute algorithms.
How to resolve this discrepancy?🧵
Alignment optimization objective implicitly assumes 𝘴𝘢𝘮𝘱𝘭𝘪𝘯𝘨 from the resulting aligned model. But we are increasingly using different and sometimes sophisticated inference-time compute algorithms.
How to resolve this discrepancy?🧵
I'll be at NeurIPS this week. Please reach out if you would like to chat!
December 9, 2024 at 9:51 PM
I'll be at NeurIPS this week. Please reach out if you would like to chat!
Reposted by Pete Shaw
New starter pack! go.bsky.app/GZ4hZzu
October 28, 2024 at 9:43 AM
New starter pack! go.bsky.app/GZ4hZzu
Reposted by Pete Shaw
Two BioML starter packs now:
Pack 1: go.bsky.app/2VWBcCd
Pack 2: go.bsky.app/Bw84Hmc
DM if you want to be included (or nominate people who should be!)
Pack 1: go.bsky.app/2VWBcCd
Pack 2: go.bsky.app/Bw84Hmc
DM if you want to be included (or nominate people who should be!)
Anybody have a bioml starter pack?
November 18, 2024 at 5:09 PM
Two BioML starter packs now:
Pack 1: go.bsky.app/2VWBcCd
Pack 2: go.bsky.app/Bw84Hmc
DM if you want to be included (or nominate people who should be!)
Pack 1: go.bsky.app/2VWBcCd
Pack 2: go.bsky.app/Bw84Hmc
DM if you want to be included (or nominate people who should be!)
Reposted by Pete Shaw
Wanted to share that Varun Godbole recently released a prompting playbook. The title says prompt tuning, but this is text prompts, not soft prompts.
github.com/varungodbole...
github.com/varungodbole...
GitHub - varungodbole/prompt-tuning-playbook: A playbook for effectively prompting post-trained LLMs
A playbook for effectively prompting post-trained LLMs - varungodbole/prompt-tuning-playbook
github.com
November 11, 2024 at 3:51 PM
Wanted to share that Varun Godbole recently released a prompting playbook. The title says prompt tuning, but this is text prompts, not soft prompts.
github.com/varungodbole...
github.com/varungodbole...
Reposted by Pete Shaw
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
November 9, 2024 at 9:13 AM
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
Getting set up on Bluesky today!
November 16, 2024 at 12:40 AM
Getting set up on Bluesky today!
Reposted by Pete Shaw
I’m pretty excited about this one!
ALTA is A Language for Transformer Analysis.
Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*.
arxiv.org/abs/2410.18077
ALTA is A Language for Transformer Analysis.
Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*.
arxiv.org/abs/2410.18077
ALTA: Compiler-Based Analysis of Transformers
We propose a new programming language called ALTA and a compiler that can map ALTA programs to Transformer weights. ALTA is inspired by RASP, a language proposed by Weiss et al. (2021), and Tracr (Lin...
arxiv.org
October 24, 2024 at 3:31 AM
I’m pretty excited about this one!
ALTA is A Language for Transformer Analysis.
Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*.
arxiv.org/abs/2410.18077
ALTA is A Language for Transformer Analysis.
Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*.
arxiv.org/abs/2410.18077