Tom Ringstrom
@noreward4u.bsky.social
Reward-Free Model-based Maximalist. High-dimensional Empowerment. Self-Preserving Autonomous Agents. Theories of intelligence grounded in compositional control.
Pinned
Tom Ringstrom
@noreward4u.bsky.social
· Nov 13
Coming over from twitter, please give me a follow! My thesis below is on the connection between compositional planning and intrinsic motivation for self-preserving agents 🙂
drive.google.com/file/d/1peG7...
drive.google.com/file/d/1peG7...
Reposted by Tom Ringstrom
What drives behavior in living organisms? And how can we design artificial agents that learn interactively?
📢 To address these, the Sensorimotor AI Journal Club is launching the "RL Debate Series"👇
w/ @elisennesh.bsky.social, @noreward4u.bsky.social, @tommasosalvatori.bsky.social
🧵[1/5]
🧠🤖🧠📈
📢 To address these, the Sensorimotor AI Journal Club is launching the "RL Debate Series"👇
w/ @elisennesh.bsky.social, @noreward4u.bsky.social, @tommasosalvatori.bsky.social
🧵[1/5]
🧠🤖🧠📈
September 17, 2025 at 4:32 PM
What drives behavior in living organisms? And how can we design artificial agents that learn interactively?
📢 To address these, the Sensorimotor AI Journal Club is launching the "RL Debate Series"👇
w/ @elisennesh.bsky.social, @noreward4u.bsky.social, @tommasosalvatori.bsky.social
🧵[1/5]
🧠🤖🧠📈
📢 To address these, the Sensorimotor AI Journal Club is launching the "RL Debate Series"👇
w/ @elisennesh.bsky.social, @noreward4u.bsky.social, @tommasosalvatori.bsky.social
🧵[1/5]
🧠🤖🧠📈
First time seeing a wild giraffe. Just chillin’ on the side of the road.
March 28, 2025 at 2:04 PM
First time seeing a wild giraffe. Just chillin’ on the side of the road.
They say don’t meet your heroes, but I traveled to South Africa and met mine and it was worth it. Stoffel the Honey Badger became a major inspiration for my PhD thesis when my advisor showed our lab a BBC show on clever animals who can solve long horizon tasks, presumably for abstract reasons.
March 24, 2025 at 9:17 PM
They say don’t meet your heroes, but I traveled to South Africa and met mine and it was worth it. Stoffel the Honey Badger became a major inspiration for my PhD thesis when my advisor showed our lab a BBC show on clever animals who can solve long horizon tasks, presumably for abstract reasons.
Reposted by Tom Ringstrom
I've been thinking recently about Bostrom's notion of instrumental convergence, and what would entail instrumental divergence. There's an obvious sense in which infinitely large time horizons and infinitely small relevant probabilities play a role in washing out potential differences.
March 13, 2025 at 10:09 AM
I've been thinking recently about Bostrom's notion of instrumental convergence, and what would entail instrumental divergence. There's an obvious sense in which infinitely large time horizons and infinitely small relevant probabilities play a role in washing out potential differences.
Unfortunately for SlipFrosty, a theory of instrumental intelligence is inseparable from a theory of normative intelligence. Abolish the value function!
January 6, 2025 at 5:53 PM
Unfortunately for SlipFrosty, a theory of instrumental intelligence is inseparable from a theory of normative intelligence. Abolish the value function!
My favorite interview from the past year, of philosopher Pete Wolfendale. Recommended to anyone interested in AI, the relationship between value, aesthetics and ethics, or anyone who wants a reason to abandon "rationality as Bayes + Utility".
www.youtube.com/watch?v=0xMc...
www.youtube.com/watch?v=0xMc...
The Gradient Podcast - Pete Wolfendale: The Revenge of Reason
YouTube video by The Gradient
www.youtube.com
December 31, 2024 at 2:11 AM
My favorite interview from the past year, of philosopher Pete Wolfendale. Recommended to anyone interested in AI, the relationship between value, aesthetics and ethics, or anyone who wants a reason to abandon "rationality as Bayes + Utility".
www.youtube.com/watch?v=0xMc...
www.youtube.com/watch?v=0xMc...
IMO, A problem with RL is that, in sparse-reward problems, value functions don’t have a general decomposition over high-dimensional transition kernels so people are trying to learn neural-net approximations to difficult-to-generalize functions from a lot of experience.
Fun ep.
Fun ep.
E61: Neurips 2024 RL meetup Hot takes: "What sucks about RL?"
What do RL researchers complain about after hours at the bar? In this "Hot takes" episode, we find out!
Recorded at The Pearl in downtown Vancouver, during the RL meetup after a day of Neurips 2024.
What do RL researchers complain about after hours at the bar? In this "Hot takes" episode, we find out!
Recorded at The Pearl in downtown Vancouver, during the RL meetup after a day of Neurips 2024.
December 26, 2024 at 5:08 PM
IMO, A problem with RL is that, in sparse-reward problems, value functions don’t have a general decomposition over high-dimensional transition kernels so people are trying to learn neural-net approximations to difficult-to-generalize functions from a lot of experience.
Fun ep.
Fun ep.
Reposted by Tom Ringstrom
Stellar new work lead by the inimitable James Whittington in Neuron that develops a new theory unifying episodic and working memory and explains diverse hippocampal and prefrontal data: www.cell.com/neuron/fullt... w/Will Dorrell, @behrenstimb.bsky.social Mohamedy El Gaby
A tale of two algorithms: Structured slots explain prefrontal sequence memory and are unified with hippocampal cognitive maps
The algorithm of the prefrontal working memory system on sequence memory tasks is not well understood, whereas it is well understood for the hippocampal episodic memory system. This work shows a mathematical duality between working and episodic sequence memory. This leads to a computational understanding of sequence working memory—as structured activity slots. This theory algorithmically explains recurrent neural network (RNN) and prefrontal representations during sequence memory tasks.
www.cell.com
December 6, 2024 at 6:47 PM
Stellar new work lead by the inimitable James Whittington in Neuron that develops a new theory unifying episodic and working memory and explains diverse hippocampal and prefrontal data: www.cell.com/neuron/fullt... w/Will Dorrell, @behrenstimb.bsky.social Mohamedy El Gaby
Reposted by Tom Ringstrom
Starter packs are making it easier to keep constructing niches, so I made this one for people broadly interested in intrinsic motivation, with some focus on RL, neuroscience and cognitive science. Haven’t found many people on Bluesky that I wanted to add here, so help me out! go.bsky.app/TPNrnpE
November 26, 2024 at 3:19 AM
Starter packs are making it easier to keep constructing niches, so I made this one for people broadly interested in intrinsic motivation, with some focus on RL, neuroscience and cognitive science. Haven’t found many people on Bluesky that I wanted to add here, so help me out! go.bsky.app/TPNrnpE
Reposted by Tom Ringstrom
I wish we could turn some of the starter-packs into a custom feed rather than following everyone.
November 25, 2024 at 5:37 PM
I wish we could turn some of the starter-packs into a custom feed rather than following everyone.
BSKY academics, lets get to know each other! Quote this & tell me: 1) a project you are working on & 2) an odd idea/theory you aren’t working on but keep thinking about
1. High-dim planning
2. What are the minimal requirements for two agents to recognize each other as acting on behalf of reasons.
1. High-dim planning
2. What are the minimal requirements for two agents to recognize each other as acting on behalf of reasons.
Bluesky academics, lets get to know each other! Quote this & tell me: 1) a project you are working on & 2) an odd idea/theory you aren’t working on but keep thinking about
1. I came to hate my work and thinking so don't do it anymore.
2.
1. I came to hate my work and thinking so don't do it anymore.
2.
Bluesky academics, lets get to know each other! Quote this & tell me: 1) a project you are working on & 2) an odd idea/theory you aren’t working on but keep thinking about
1. Convincing everyone that everything is luck, all the way down.
2. LLM’s can reason and understand in the external sense.
1. Convincing everyone that everything is luck, all the way down.
2. LLM’s can reason and understand in the external sense.
November 17, 2024 at 7:14 PM
BSKY academics, lets get to know each other! Quote this & tell me: 1) a project you are working on & 2) an odd idea/theory you aren’t working on but keep thinking about
1. High-dim planning
2. What are the minimal requirements for two agents to recognize each other as acting on behalf of reasons.
1. High-dim planning
2. What are the minimal requirements for two agents to recognize each other as acting on behalf of reasons.
Reposted by Tom Ringstrom
The concept of system state is fundamental in theories of control and computation. I wrote down a few thoughts on the history of this idea and its connections to rationalist and empiricist perspectives in philosophy.
Seeming Like a State
What's Nerode-equivalent to the past is prologue.
realizable.substack.com
November 15, 2024 at 11:03 PM
The concept of system state is fundamental in theories of control and computation. I wrote down a few thoughts on the history of this idea and its connections to rationalist and empiricist perspectives in philosophy.
Coming over from twitter, please give me a follow! My thesis below is on the connection between compositional planning and intrinsic motivation for self-preserving agents 🙂
drive.google.com/file/d/1peG7...
drive.google.com/file/d/1peG7...
November 13, 2024 at 9:08 PM
Coming over from twitter, please give me a follow! My thesis below is on the connection between compositional planning and intrinsic motivation for self-preserving agents 🙂
drive.google.com/file/d/1peG7...
drive.google.com/file/d/1peG7...