Dylan Foster 🐢
@djfoster.bsky.social
Principal Researcher in AI/ML/RL Theory @ Microsoft Research NE/NYC. Previously @ MIT, Cornell. http://dylanfoster.net
RL Theory Lecture Notes: https://arxiv.org/abs/2312.16730
RL Theory Lecture Notes: https://arxiv.org/abs/2312.16730
Pinned
Dylan Foster 🐢
@djfoster.bsky.social
· Nov 21
As my first post on this platform, allow me to advertise the RL theory lecture notes I have been developing with Sasha Rakhlin: arxiv.org/abs/2312.16730
(shameless repost of my pinned tweet)
(shameless repost of my pinned tweet)
The coverage principle: How pre-training enables post-training
New preprint where we look at the mechanisms through which next-token prediction produces models that succeed at downstream tasks.
The answer involves a metric we call the "coverage profile", not cross-entropy.
New preprint where we look at the mechanisms through which next-token prediction produces models that succeed at downstream tasks.
The answer involves a metric we call the "coverage profile", not cross-entropy.
October 25, 2025 at 4:20 PM
The coverage principle: How pre-training enables post-training
New preprint where we look at the mechanisms through which next-token prediction produces models that succeed at downstream tasks.
The answer involves a metric we call the "coverage profile", not cross-entropy.
New preprint where we look at the mechanisms through which next-token prediction produces models that succeed at downstream tasks.
The answer involves a metric we call the "coverage profile", not cross-entropy.
Reposted by Dylan Foster 🐢
The new call for Motwani postdocs application is now open!
academicjobsonline.org/ajo/jobs/30865
BTW-
Not quite ready for a postdoc? We updated the TCS Masters programs spreadsheet:
www.cs.princeton.edu/~smattw/mast...
Any career stage and in the (SF) Bay Area?
Save the date for TOCA-SV on 11/7!
academicjobsonline.org/ajo/jobs/30865
BTW-
Not quite ready for a postdoc? We updated the TCS Masters programs spreadsheet:
www.cs.princeton.edu/~smattw/mast...
Any career stage and in the (SF) Bay Area?
Save the date for TOCA-SV on 11/7!
Stanford University, Computer Science/Theory Lab/Stanford University
Job #AJO30865, Postdoc in Theoretical Computer Science at Stanford, Computer Science/Theory Lab/Stanford University, Stanford University, Stanford, California, US
academicjobsonline.org
October 13, 2025 at 8:42 PM
The new call for Motwani postdocs application is now open!
academicjobsonline.org/ajo/jobs/30865
BTW-
Not quite ready for a postdoc? We updated the TCS Masters programs spreadsheet:
www.cs.princeton.edu/~smattw/mast...
Any career stage and in the (SF) Bay Area?
Save the date for TOCA-SV on 11/7!
academicjobsonline.org/ajo/jobs/30865
BTW-
Not quite ready for a postdoc? We updated the TCS Masters programs spreadsheet:
www.cs.princeton.edu/~smattw/mast...
Any career stage and in the (SF) Bay Area?
Save the date for TOCA-SV on 11/7!
Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking.
A totally new framework based on ~backtracking~ for using process verifiers to guide inference, w/ connections to approximate counting/sampling in theoretical CS.
Paper: www.arxiv.org/abs/2510.03149
A totally new framework based on ~backtracking~ for using process verifiers to guide inference, w/ connections to approximate counting/sampling in theoretical CS.
Paper: www.arxiv.org/abs/2510.03149
October 12, 2025 at 4:26 PM
Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking.
A totally new framework based on ~backtracking~ for using process verifiers to guide inference, w/ connections to approximate counting/sampling in theoretical CS.
Paper: www.arxiv.org/abs/2510.03149
A totally new framework based on ~backtracking~ for using process verifiers to guide inference, w/ connections to approximate counting/sampling in theoretical CS.
Paper: www.arxiv.org/abs/2510.03149
MSR NYC is hiring spring and summer interns in AI/ML/RL!
Apply here: jobs.careers.microsoft.com/global/en/jo...
Apply here: jobs.careers.microsoft.com/global/en/jo...
Microsoft Research Lab - New York City - Microsoft Research
Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.
www.microsoft.com
October 2, 2025 at 8:58 PM
MSR NYC is hiring spring and summer interns in AI/ML/RL!
Apply here: jobs.careers.microsoft.com/global/en/jo...
Apply here: jobs.careers.microsoft.com/global/en/jo...
Reposted by Dylan Foster 🐢
🚨Microsoft Research NYC is hiring🚨
We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis.
Links to apply: aka.ms/msrnyc-jobs
We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis.
Links to apply: aka.ms/msrnyc-jobs
Microsoft Research Lab - New York City - Microsoft Research
Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.
aka.ms
September 18, 2025 at 2:37 PM
🚨Microsoft Research NYC is hiring🚨
We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis.
Links to apply: aka.ms/msrnyc-jobs
We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis.
Links to apply: aka.ms/msrnyc-jobs
Microsoft Research New York City (www.microsoft.com/en-us/resear...) is seeking applicants for multiple Postdoctoral Researcher positions in ML/AI!
These are positions for up to 2 years, starting in July 2026.
Application deadline: October 22, 2025
These are positions for up to 2 years, starting in July 2026.
Application deadline: October 22, 2025
Microsoft Research Lab - New York City - Microsoft Research
Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.
www.microsoft.com
September 12, 2025 at 2:57 PM
Microsoft Research New York City (www.microsoft.com/en-us/resear...) is seeking applicants for multiple Postdoctoral Researcher positions in ML/AI!
These are positions for up to 2 years, starting in July 2026.
Application deadline: October 22, 2025
These are positions for up to 2 years, starting in July 2026.
Application deadline: October 22, 2025
Quick reminder: The deadline for our workshop on Foundations of Reasoning in Language Models (FoRLM) at NeurIPS 2025 is next Wednesday, Sept 3!
Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025!
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
August 27, 2025 at 6:51 PM
Quick reminder: The deadline for our workshop on Foundations of Reasoning in Language Models (FoRLM) at NeurIPS 2025 is next Wednesday, Sept 3!
Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025!
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
August 11, 2025 at 3:40 PM
Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025!
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.
📆 Deadline: Sept 3, 2025
For those at ICML, Audrey will be presenting this paper at the 4:30pm poster session this afternoon! West Exhibition Hall B2-B3 W-1009
July 15, 2025 at 3:59 PM
For those at ICML, Audrey will be presenting this paper at the 4:30pm poster session this afternoon! West Exhibition Hall B2-B3 W-1009
Reposted by Dylan Foster 🐢
ICML's election for their board of directors has begun. I've thrown my hat in the ring. Please consider voting for Gautam Kamath.
I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.
I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.
June 30, 2025 at 12:44 PM
ICML's election for their board of directors has begun. I've thrown my hat in the ring. Please consider voting for Gautam Kamath.
I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.
I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.
Reposted by Dylan Foster 🐢
This week's #PaperILike is "The Power of Resets in Online Reinforcement Learning" (Mhammedi et al., 2024).
If you're doing RL in sim, why not use the sim to its full potential? Reset to any state! (gym.Env.reset() is not all we need.)
PDF: arxiv.org/abs/2404.15417
If you're doing RL in sim, why not use the sim to its full potential? Reset to any state! (gym.Env.reset() is not all we need.)
PDF: arxiv.org/abs/2404.15417
The Power of Resets in Online Reinforcement Learning
Simulators are a pervasive tool in reinforcement learning, but most existing algorithms cannot efficiently exploit simulator access -- particularly in high-dimensional domains that require general fun...
arxiv.org
June 29, 2025 at 1:08 PM
This week's #PaperILike is "The Power of Resets in Online Reinforcement Learning" (Mhammedi et al., 2024).
If you're doing RL in sim, why not use the sim to its full potential? Reset to any state! (gym.Env.reset() is not all we need.)
PDF: arxiv.org/abs/2404.15417
If you're doing RL in sim, why not use the sim to its full potential? Reset to any state! (gym.Env.reset() is not all we need.)
PDF: arxiv.org/abs/2404.15417
Reposted by Dylan Foster 🐢
📣Join us at COLT 2025 in Lyon for a community event!
📅When: Mon, June 30 | 16:00 CET
What: Fireside chat w/ Peter Bartlett & Vitaly Feldman on communicating a research agenda, followed by mentorship roundtable to practice elevator pitches & mingle w/ COLT community!
let-all.com/colt25.html
📅When: Mon, June 30 | 16:00 CET
What: Fireside chat w/ Peter Bartlett & Vitaly Feldman on communicating a research agenda, followed by mentorship roundtable to practice elevator pitches & mingle w/ COLT community!
let-all.com/colt25.html
June 24, 2025 at 6:22 PM
📣Join us at COLT 2025 in Lyon for a community event!
📅When: Mon, June 30 | 16:00 CET
What: Fireside chat w/ Peter Bartlett & Vitaly Feldman on communicating a research agenda, followed by mentorship roundtable to practice elevator pitches & mingle w/ COLT community!
let-all.com/colt25.html
📅When: Mon, June 30 | 16:00 CET
What: Fireside chat w/ Peter Bartlett & Vitaly Feldman on communicating a research agenda, followed by mentorship roundtable to practice elevator pitches & mingle w/ COLT community!
let-all.com/colt25.html
Reposted by Dylan Foster 🐢
Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please!
Full posting to come in a bit.
Full posting to come in a bit.
Robust Autonomy Emerges from Self-Play
Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic drivi...
arxiv.org
June 21, 2025 at 5:14 PM
Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please!
Full posting to come in a bit.
Full posting to come in a bit.
Reposted by Dylan Foster 🐢
At the IDEAL annual meeting and saw this paper presented. Basically: reducing length of chain of thought LLM computations by deleting intermediate computations, more like classical functional programming where only function call and return values are important.
arxiv.org/abs/2503.14337
arxiv.org/abs/2503.14337
PENCIL: Long Thoughts with Short Memory
While recent works (e.g. o1, DeepSeek R1) have demonstrated great promise of using long Chain-of-Thought (CoT) to improve reasoning capabilities of language models, scaling it up during test-time is c...
arxiv.org
June 9, 2025 at 2:54 PM
At the IDEAL annual meeting and saw this paper presented. Basically: reducing length of chain of thought LLM computations by deleting intermediate computations, more like classical functional programming where only function call and return values are important.
arxiv.org/abs/2503.14337
arxiv.org/abs/2503.14337
Reposted by Dylan Foster 🐢
Dhruv Rohatgi will be giving a lecture on our recent work on comp-stat tradeoffs in next-token prediction at the RL Theory virtual seminar series (rl-theory.bsky.social) tomorrow at 2pm EST! Should be a fun talk---come check it out!!
May 26, 2025 at 7:19 PM
Dhruv Rohatgi will be giving a lecture on our recent work on comp-stat tradeoffs in next-token prediction at the RL Theory virtual seminar series (rl-theory.bsky.social) tomorrow at 2pm EST! Should be a fun talk---come check it out!!
Reposted by Dylan Foster 🐢
Later today, Sikata and Marcel will talk about their recent work on oracle-efficient RL with ensembles. Join us!
May 20, 2025 at 3:48 PM
Later today, Sikata and Marcel will talk about their recent work on oracle-efficient RL with ensembles. Join us!
The abstract submission deadline for FoPt has been extended to the 21st of May (11:59pm UTC).
Submission website: openreview.net/group?id=lea...
Submission website: openreview.net/group?id=lea...
Announcing the first workshop on Foundations of Post-Training (FoPT) at COLT 2025!
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
May 19, 2025 at 6:27 PM
The abstract submission deadline for FoPt has been extended to the 21st of May (11:59pm UTC).
Submission website: openreview.net/group?id=lea...
Submission website: openreview.net/group?id=lea...
Reposted by Dylan Foster 🐢
Announcing the first workshop on Foundations of Post-Training (FoPT) at COLT 2025!
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
May 9, 2025 at 5:10 PM
Announcing the first workshop on Foundations of Post-Training (FoPT) at COLT 2025!
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
Announcing the first workshop on Foundations of Post-Training (FoPT) at COLT 2025!
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
May 9, 2025 at 5:10 PM
Announcing the first workshop on Foundations of Post-Training (FoPT) at COLT 2025!
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
📝 Soliciting abstracts/posters exploring theoretical & practical aspects of post-training and RL with language models!
🗓️ Deadline: May 19, 2025
Is Best-of-N really the best we can do for language model inference?
New paper (appearing at ICML) led by the amazing Audrey Huang (ahahaudrey.bsky.social) with Adam Block, Qinghua Liu, Nan Jiang, and Akshay Krishnamurthy (akshaykr.bsky.social).
1/11
New paper (appearing at ICML) led by the amazing Audrey Huang (ahahaudrey.bsky.social) with Adam Block, Qinghua Liu, Nan Jiang, and Akshay Krishnamurthy (akshaykr.bsky.social).
1/11
May 3, 2025 at 5:40 PM
Is Best-of-N really the best we can do for language model inference?
New paper (appearing at ICML) led by the amazing Audrey Huang (ahahaudrey.bsky.social) with Adam Block, Qinghua Liu, Nan Jiang, and Akshay Krishnamurthy (akshaykr.bsky.social).
1/11
New paper (appearing at ICML) led by the amazing Audrey Huang (ahahaudrey.bsky.social) with Adam Block, Qinghua Liu, Nan Jiang, and Akshay Krishnamurthy (akshaykr.bsky.social).
1/11
Reposted by Dylan Foster 🐢
Last seminars before the summer break:
04/29: Max Simchowitz (CMU)
05/06: Jeongyeol Kwon (Univ. of Widsconsin-Madison)
05/20: Sikata Sengupta & Marcel Hussing (Univ. of Pennsylvania)
05/27: Dhruv Rohatgi (MIT)
06/03: David Janz (Univ. of Oxford)
06/10: Nneka Okolo (MIT)
04/29: Max Simchowitz (CMU)
05/06: Jeongyeol Kwon (Univ. of Widsconsin-Madison)
05/20: Sikata Sengupta & Marcel Hussing (Univ. of Pennsylvania)
05/27: Dhruv Rohatgi (MIT)
06/03: David Janz (Univ. of Oxford)
06/10: Nneka Okolo (MIT)
April 16, 2025 at 5:20 PM
Last seminars before the summer break:
04/29: Max Simchowitz (CMU)
05/06: Jeongyeol Kwon (Univ. of Widsconsin-Madison)
05/20: Sikata Sengupta & Marcel Hussing (Univ. of Pennsylvania)
05/27: Dhruv Rohatgi (MIT)
06/03: David Janz (Univ. of Oxford)
06/10: Nneka Okolo (MIT)
04/29: Max Simchowitz (CMU)
05/06: Jeongyeol Kwon (Univ. of Widsconsin-Madison)
05/20: Sikata Sengupta & Marcel Hussing (Univ. of Pennsylvania)
05/27: Dhruv Rohatgi (MIT)
06/03: David Janz (Univ. of Oxford)
06/10: Nneka Okolo (MIT)
Reposted by Dylan Foster 🐢
What is the place of exploration in today's AI landscape and in which settings can exploration algorithms address current open challenges?
Join us to discuss this at our exciting workshop at @icmlconf.bsky.social 2025: EXAIT!
exait-workshop.github.io
#ICML2025
Join us to discuss this at our exciting workshop at @icmlconf.bsky.social 2025: EXAIT!
exait-workshop.github.io
#ICML2025
April 17, 2025 at 5:53 AM
What is the place of exploration in today's AI landscape and in which settings can exploration algorithms address current open challenges?
Join us to discuss this at our exciting workshop at @icmlconf.bsky.social 2025: EXAIT!
exait-workshop.github.io
#ICML2025
Join us to discuss this at our exciting workshop at @icmlconf.bsky.social 2025: EXAIT!
exait-workshop.github.io
#ICML2025
Reinforcement learning has led to amazing breakthroughs in reasoning (e.g., R1), but can it discover truly new behaviors not already present in the base model?
A new paper with Zak Mhammedi and Dhruv Rohatgi:
The Computational Role of the Base Model in Exploration
arxiv.org/abs/2503.07453
A new paper with Zak Mhammedi and Dhruv Rohatgi:
The Computational Role of the Base Model in Exploration
arxiv.org/abs/2503.07453
March 27, 2025 at 5:28 PM
Reinforcement learning has led to amazing breakthroughs in reasoning (e.g., R1), but can it discover truly new behaviors not already present in the base model?
A new paper with Zak Mhammedi and Dhruv Rohatgi:
The Computational Role of the Base Model in Exploration
arxiv.org/abs/2503.07453
A new paper with Zak Mhammedi and Dhruv Rohatgi:
The Computational Role of the Base Model in Exploration
arxiv.org/abs/2503.07453
Reposted by Dylan Foster 🐢
Join us tomorrow to attend Vlad's presentation! Related to the seminar from last week, but this time in the offline setting.
Tuesday March 25, 6 PM UTC.
Tuesday March 25, 6 PM UTC.
March 24, 2025 at 1:07 PM
Join us tomorrow to attend Vlad's presentation! Related to the seminar from last week, but this time in the offline setting.
Tuesday March 25, 6 PM UTC.
Tuesday March 25, 6 PM UTC.