@cogscikid.bsky.social
Kempner Institute research fellow @Harvard interested in scaling up (deep) reinforcement learning theories of human cognition
prev: deepmind, umich, msr
https://cogscikid.com/
prev: deepmind, umich, msr
https://cogscikid.com/
Excited to share recent work on an AI model that doubles as a scalable cognitive theory.
Basic idea: when learning one task, "preplay" other tasks to learn implicit goal-oriented maps that support fast performance later.
This improves AI and predicts human behavior in naturalistic experiments!
Basic idea: when learning one task, "preplay" other tasks to learn implicit goal-oriented maps that support fast performance later.
This improves AI and predicts human behavior in naturalistic experiments!
September 10, 2025 at 2:37 PM
Excited to share recent work on an AI model that doubles as a scalable cognitive theory.
Basic idea: when learning one task, "preplay" other tasks to learn implicit goal-oriented maps that support fast performance later.
This improves AI and predicts human behavior in naturalistic experiments!
Basic idea: when learning one task, "preplay" other tasks to learn implicit goal-oriented maps that support fast performance later.
This improves AI and predicts human behavior in naturalistic experiments!
Main result 3: we speculate that this may have unexpected benefits for human generalization.
We present AI simulations where Multitask Preplay improves generalization of complex, long-horizon behaviors to 10,000 unique new environments when they share subtask co-occurrence structure.
We present AI simulations where Multitask Preplay improves generalization of complex, long-horizon behaviors to 10,000 unique new environments when they share subtask co-occurrence structure.
July 12, 2025 at 4:20 PM
Main result 3: we speculate that this may have unexpected benefits for human generalization.
We present AI simulations where Multitask Preplay improves generalization of complex, long-horizon behaviors to 10,000 unique new environments when they share subtask co-occurrence structure.
We present AI simulations where Multitask Preplay improves generalization of complex, long-horizon behaviors to 10,000 unique new environments when they share subtask co-occurrence structure.
Main result 2: we generalize these predictions to Craftax, a partially observable, 2D minecraft domain.
Here, we once again find evidence that people preplay completion of tasks that were accessible but unpursued, but now in a much larger world where generalization to new tasks is much harder.
Here, we once again find evidence that people preplay completion of tasks that were accessible but unpursued, but now in a much larger world where generalization to new tasks is much harder.
July 12, 2025 at 4:20 PM
Main result 2: we generalize these predictions to Craftax, a partially observable, 2D minecraft domain.
Here, we once again find evidence that people preplay completion of tasks that were accessible but unpursued, but now in a much larger world where generalization to new tasks is much harder.
Here, we once again find evidence that people preplay completion of tasks that were accessible but unpursued, but now in a much larger world where generalization to new tasks is much harder.
Main result 1: Across 4 experiments, we find evidence that people preplay completion of tasks that were accessible but unpursued, even if they don't know those tasks will come up later on.
July 12, 2025 at 4:20 PM
Main result 1: Across 4 experiments, we find evidence that people preplay completion of tasks that were accessible but unpursued, even if they don't know those tasks will come up later on.
Key idea behind Multitask Preplay: when people engage in replay of one task, they might "preplay" completion of another task and leverage temporal-difference learning to cache the results into a neural network, enabling fast, automatic behavior for that task later on.
arxiv.org/abs/2507.05561
arxiv.org/abs/2507.05561
July 12, 2025 at 4:20 PM
Key idea behind Multitask Preplay: when people engage in replay of one task, they might "preplay" completion of another task and leverage temporal-difference learning to cache the results into a neural network, enabling fast, automatic behavior for that task later on.
arxiv.org/abs/2507.05561
arxiv.org/abs/2507.05561
Excited to share a new project spanning cognitive science and AI where we develop a novel deep reinforcement learning model---Multitask Preplay---that explains how people generalize to new tasks that were previously accessible but unpursued.
July 12, 2025 at 4:20 PM
Excited to share a new project spanning cognitive science and AI where we develop a novel deep reinforcement learning model---Multitask Preplay---that explains how people generalize to new tasks that were previously accessible but unpursued.
NiceWebRL allows you to use the *exact same* environment for both developing computational models (AI or cognitive) and for conducting online human subject experiments
June 17, 2025 at 5:09 PM
NiceWebRL allows you to use the *exact same* environment for both developing computational models (AI or cognitive) and for conducting online human subject experiments
We discuss different avenues for garnering understanding: from looking at data as a source for explanation, to mechanistic interpretability, to developing reductive explanations that link task-performing models to the natural behaviors we observe.
June 16, 2025 at 7:35 PM
We discuss different avenues for garnering understanding: from looking at data as a source for explanation, to mechanistic interpretability, to developing reductive explanations that link task-performing models to the natural behaviors we observe.
We first describe how cognitive science can take lessons from the engineering practices of machine learning to better develop generalizable models.
In particular, we emphasize frictionless reproducibility, generalizability-focused model development , and the utility of hypothesis-driven benchmarks.
In particular, we emphasize frictionless reproducibility, generalizability-focused model development , and the utility of hypothesis-driven benchmarks.
June 16, 2025 at 7:35 PM
We first describe how cognitive science can take lessons from the engineering practices of machine learning to better develop generalizable models.
In particular, we emphasize frictionless reproducibility, generalizability-focused model development , and the utility of hypothesis-driven benchmarks.
In particular, we emphasize frictionless reproducibility, generalizability-focused model development , and the utility of hypothesis-driven benchmarks.
Afterwards, we motivate learning-based models by highlighting how AI research has revealed that learning with naturalistic data can lead to qualitatively different patterns of behavior and generalization compared to simplified settings. This may have profound implications for our cognitive theories.
June 16, 2025 at 7:35 PM
Afterwards, we motivate learning-based models by highlighting how AI research has revealed that learning with naturalistic data can lead to qualitatively different patterns of behavior and generalization compared to simplified settings. This may have profound implications for our cognitive theories.
We first motivate the need for naturalistic experimental paradigms by showcasing examples from neuroscience, cognitive science, and AI where learning, behavior, and brain computation manifest differently in naturalistic experimental conditions.
June 16, 2025 at 7:31 PM
We first motivate the need for naturalistic experimental paradigms by showcasing examples from neuroscience, cognitive science, and AI where learning, behavior, and brain computation manifest differently in naturalistic experimental conditions.
For example, thanks to technologies like generative AI and augmented reality, we can now create *synthetic*, parametrically controllable naturalistic environments.
June 16, 2025 at 7:30 PM
For example, thanks to technologies like generative AI and augmented reality, we can now create *synthetic*, parametrically controllable naturalistic environments.
Naturalistic computational cognitive science is framework for theory-driven cognitive science that aims to develop theories that can predict and explain human behavior in increasingly naturalistic conditions.
June 16, 2025 at 7:30 PM
Naturalistic computational cognitive science is framework for theory-driven cognitive science that aims to develop theories that can predict and explain human behavior in increasingly naturalistic conditions.
In this paper, we specify a research strategy for building models and theories that aim to characterize natural behavior across the settings it manifests.
To help readers, we provide an outline of our complete argument as our first figure.
arxiv.org/abs/2502.20349
To help readers, we provide an outline of our complete argument as our first figure.
arxiv.org/abs/2502.20349
June 16, 2025 at 7:30 PM
In this paper, we specify a research strategy for building models and theories that aim to characterize natural behavior across the settings it manifests.
To help readers, we provide an outline of our complete argument as our first figure.
arxiv.org/abs/2502.20349
To help readers, we provide an outline of our complete argument as our first figure.
arxiv.org/abs/2502.20349
Excited to share this project specifying a research direction I think will be particularly fruitful for theory-driven cognitive science that aims to explain natural behavior!
We're calling this direction "Naturalistic Computational Cognitive Science"
We're calling this direction "Naturalistic Computational Cognitive Science"
June 16, 2025 at 7:30 PM
Excited to share this project specifying a research direction I think will be particularly fruitful for theory-driven cognitive science that aims to explain natural behavior!
We're calling this direction "Naturalistic Computational Cognitive Science"
We're calling this direction "Naturalistic Computational Cognitive Science"
Excited to return to my PhD institution, UMich, to give this talk next week!
ece.engin.umich.edu/event/invite...
ece.engin.umich.edu/event/invite...
February 20, 2025 at 9:42 PM
Excited to return to my PhD institution, UMich, to give this talk next week!
ece.engin.umich.edu/event/invite...
ece.engin.umich.edu/event/invite...
The second example we have allows two people to coordinate on "cooking" tasks in an environment called "overcooked".
Importantly, this supports the popular JaxMARL (Multi-agent RL) codebase and all of its environments
Importantly, this supports the popular JaxMARL (Multi-agent RL) codebase and all of its environments
February 10, 2025 at 5:58 PM
The second example we have allows two people to coordinate on "cooking" tasks in an environment called "overcooked".
Importantly, this supports the popular JaxMARL (Multi-agent RL) codebase and all of its environments
Importantly, this supports the popular JaxMARL (Multi-agent RL) codebase and all of its environments
Right now, I'm setting up an "examples" folder with (ideally) domains + algorithms researchers already care about.
Are there any examples the community would like to see?
The first example we have allows a human to control an agent in a 2D minecraft environment known as "crafter"
Are there any examples the community would like to see?
The first example we have allows a human to control an agent in a 2D minecraft environment known as "crafter"
February 10, 2025 at 5:58 PM
Right now, I'm setting up an "examples" folder with (ideally) domains + algorithms researchers already care about.
Are there any examples the community would like to see?
The first example we have allows a human to control an agent in a 2D minecraft environment known as "crafter"
Are there any examples the community would like to see?
The first example we have allows a human to control an agent in a 2D minecraft environment known as "crafter"
nicewebrl supports a growing set of environments (50+) developed by the research community
February 10, 2025 at 5:58 PM
nicewebrl supports a growing set of environments (50+) developed by the research community
Are you interested in comparing modern AI/RL agents to Humans?
To make it easier for researchers spanning cognitive science, multi-agent RL, and HCI to do this, I've made nicewebrl
github.com/wcarvalho/ni...
To make it easier for researchers spanning cognitive science, multi-agent RL, and HCI to do this, I've made nicewebrl
github.com/wcarvalho/ni...
February 10, 2025 at 5:58 PM
Are you interested in comparing modern AI/RL agents to Humans?
To make it easier for researchers spanning cognitive science, multi-agent RL, and HCI to do this, I've made nicewebrl
github.com/wcarvalho/ni...
To make it easier for researchers spanning cognitive science, multi-agent RL, and HCI to do this, I've made nicewebrl
github.com/wcarvalho/ni...