Edward Grefenstette
@egrefen.bsky.social
FR/US/GB AI/ML Person, Director of Research at Google DeepMind, Honorary Professor at UCL DARK, ELLIS Fellow. Ex Oxford CS, Meta AI, Cohere.
Pinned
Edward Grefenstette
@egrefen.bsky.social
· Dec 30
🧵 As 2024 wraps up, please pardon my usual self-indulgence in tweeting about the year gone by. 🧵
This will be a reasonably short one... OR WILL IT? [1/17]
This will be a reasonably short one... OR WILL IT? [1/17]
Do you have a PhD (or equivalent) or will have one in the coming months (i.e. 2-3 months away from graduating)? Do you want to help build open-ended agents that help humans do humans things better, rather than replace them? We're hiring 1-2 Research Scientists! Check the 🧵👇
July 21, 2025 at 2:21 PM
Do you have a PhD (or equivalent) or will have one in the coming months (i.e. 2-3 months away from graduating)? Do you want to help build open-ended agents that help humans do humans things better, rather than replace them? We're hiring 1-2 Research Scientists! Check the 🧵👇
FYI this posting for a research scientist position in the autonomous assistants team at Google DeepMind will be open for a little under a week, as of today. Please consider applying if you are interested and qualify. See post for details, or ask questions here.
Our team in London is hiring a research scientist! If you want to come work with a wonderful group of researchers on investigating the frontiers of autonomous open-ended agents that help humans be better at doing things we love, come have a look. Link in post below 👇
March 25, 2025 at 3:12 PM
FYI this posting for a research scientist position in the autonomous assistants team at Google DeepMind will be open for a little under a week, as of today. Please consider applying if you are interested and qualify. See post for details, or ask questions here.
Our team in London is hiring a research scientist! If you want to come work with a wonderful group of researchers on investigating the frontiers of autonomous open-ended agents that help humans be better at doing things we love, come have a look. Link in post below 👇
March 18, 2025 at 4:01 PM
Our team in London is hiring a research scientist! If you want to come work with a wonderful group of researchers on investigating the frontiers of autonomous open-ended agents that help humans be better at doing things we love, come have a look. Link in post below 👇
🧵 As 2024 wraps up, please pardon my usual self-indulgence in tweeting about the year gone by. 🧵
This will be a reasonably short one... OR WILL IT? [1/17]
This will be a reasonably short one... OR WILL IT? [1/17]
December 30, 2024 at 10:47 PM
🧵 As 2024 wraps up, please pardon my usual self-indulgence in tweeting about the year gone by. 🧵
This will be a reasonably short one... OR WILL IT? [1/17]
This will be a reasonably short one... OR WILL IT? [1/17]
Merry Christmas (eve), you filthy animal(s).
December 24, 2024 at 5:13 PM
Merry Christmas (eve), you filthy animal(s).
Researchers: be constructively skeptical about LLMs. Find where they don't work by building with them. Find out if the failure is systemic or just transient. This way, you're best positioned to build what's next, or, if they keep working, to benefit from their growth.
December 9, 2024 at 12:18 PM
Researchers: be constructively skeptical about LLMs. Find where they don't work by building with them. Find out if the failure is systemic or just transient. This way, you're best positioned to build what's next, or, if they keep working, to benefit from their growth.
Seek novelty in what you do, how you do it, and who you do it with. I feel part of happiness lies in committing to these things, but not obsessively overcommitting to just one of these things.
December 2, 2024 at 12:16 AM
Seek novelty in what you do, how you do it, and who you do it with. I feel part of happiness lies in committing to these things, but not obsessively overcommitting to just one of these things.
Multi-agent peeps: are there any *-MDP variants where there is more than one agent, but exactly one agent is acting on the environment at each time step? Not in the sense of "we take turns" (although I guess it's a special case) but more in the sense that the agents decide who gets to act...
November 25, 2024 at 5:31 PM
Multi-agent peeps: are there any *-MDP variants where there is more than one agent, but exactly one agent is acting on the environment at each time step? Not in the sense of "we take turns" (although I guess it's a special case) but more in the sense that the agents decide who gets to act...
Reposted by Edward Grefenstette
🚨 LLMs can learn to reason from procedural knowledge in pretraining data! 🚨 I particularly enjoy research where the evidence contradicts our initial hypothesis. If you're interested in LLM reasoning, check out the 60+ pages of in-depth work at arxiv.org/abs/2411.12580
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
November 20, 2024 at 5:28 PM
🚨 LLMs can learn to reason from procedural knowledge in pretraining data! 🚨 I particularly enjoy research where the evidence contradicts our initial hypothesis. If you're interested in LLM reasoning, check out the 60+ pages of in-depth work at arxiv.org/abs/2411.12580
“LLMs can/can’t reason” — whatever you think, they clearly can solve some reasoning problems, but how do they learn to do this? Is the dependency on the training data measurable, relative to factual knowledge? Does this tell us something about their abilities? Find out here!
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
November 20, 2024 at 6:38 PM
“LLMs can/can’t reason” — whatever you think, they clearly can solve some reasoning problems, but how do they learn to do this? Is the dependency on the training data measurable, relative to factual knowledge? Does this tell us something about their abilities? Find out here!
Reposted by Edward Grefenstette
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
November 20, 2024 at 4:35 PM
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Reposted by Edward Grefenstette
Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr Locatelli, Robert Kirk, Tim Rockt\"aschel, Edward Grefenstette, Max Bartolo
Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
https://arxiv.org/abs/2411.12580
Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
https://arxiv.org/abs/2411.12580
November 20, 2024 at 7:01 AM
Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr Locatelli, Robert Kirk, Tim Rockt\"aschel, Edward Grefenstette, Max Bartolo
Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
https://arxiv.org/abs/2411.12580
Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
https://arxiv.org/abs/2411.12580
Is there some way to stop Bluesky from popping a notification on my phone every time I get a follower?
November 20, 2024 at 10:53 AM
Is there some way to stop Bluesky from popping a notification on my phone every time I get a follower?
🌶️(?) take: Agents are somehow hot right because people realized that LLM output can be interpreted as a DSL which directs side effects in the world (e.g. tool calls) rather than just returning text in a chat/autocomplete sense. What are the open challenges? A 🧵... [1/11]
November 19, 2024 at 9:32 AM
🌶️(?) take: Agents are somehow hot right because people realized that LLM output can be interpreted as a DSL which directs side effects in the world (e.g. tool calls) rather than just returning text in a chat/autocomplete sense. What are the open challenges? A 🧵... [1/11]
Is there some good way to selectively crosspost to X and Bluesky, e.g. draft a post somewhere central, and then just post to one/the other/both with a keypress or click?
Obviously I can just copy/paste... maybe that's the easiest way.
Obviously I can just copy/paste... maybe that's the easiest way.
November 18, 2024 at 3:20 PM
Is there some good way to selectively crosspost to X and Bluesky, e.g. draft a post somewhere central, and then just post to one/the other/both with a keypress or click?
Obviously I can just copy/paste... maybe that's the easiest way.
Obviously I can just copy/paste... maybe that's the easiest way.
Deep down, everything is a minmax game. We'll get to AGI (whatever that means for you) by building better minmax objectives.
November 18, 2024 at 3:18 PM
Deep down, everything is a minmax game. We'll get to AGI (whatever that means for you) by building better minmax objectives.
So are we mainly shitposting here too or should I reserve Bluesky for balanced takes on ML and leave the spicy takes (mainly rage about politics) for Twitter?
November 17, 2024 at 9:38 PM
So are we mainly shitposting here too or should I reserve Bluesky for balanced takes on ML and leave the spicy takes (mainly rage about politics) for Twitter?
What’s good, Bluesky?
November 17, 2024 at 9:37 PM
What’s good, Bluesky?
🚨 JOB ALERT 🚨
We're hiring research scientists/engineers to conduct research on next-gen assistant technologies to power increasingly autonomous agents which strive to support humans
Research Scientist: boards.greenhouse.io/deepmind/job...
Research Engineer: boards.greenhouse.io/deepmind/job...
We're hiring research scientists/engineers to conduct research on next-gen assistant technologies to power increasingly autonomous agents which strive to support humans
Research Scientist: boards.greenhouse.io/deepmind/job...
Research Engineer: boards.greenhouse.io/deepmind/job...
September 12, 2023 at 1:46 PM
🚨 JOB ALERT 🚨
We're hiring research scientists/engineers to conduct research on next-gen assistant technologies to power increasingly autonomous agents which strive to support humans
Research Scientist: boards.greenhouse.io/deepmind/job...
Research Engineer: boards.greenhouse.io/deepmind/job...
We're hiring research scientists/engineers to conduct research on next-gen assistant technologies to power increasingly autonomous agents which strive to support humans
Research Scientist: boards.greenhouse.io/deepmind/job...
Research Engineer: boards.greenhouse.io/deepmind/job...
Here we go again.
August 26, 2023 at 6:48 AM
Here we go again.