Marlos C. Machado
@marloscmachado.bsky.social
Assistant Professor at the University of Alberta. Amii Fellow, Canada CIFAR AI chair. Machine learning researcher. All things reinforcement learning.
📍 Edmonton, Canada 🇨🇦
🔗 https://webdocs.cs.ualberta.ca/~machado/
🗓️ Joined November, 2024
📍 Edmonton, Canada 🇨🇦
🔗 https://webdocs.cs.ualberta.ca/~machado/
🗓️ Joined November, 2024
"Canada Impact+ Research Chairs program—a new $1 billion investment that will provide Canadian institutions the opportunity to recruit top-tier international researchers with expertise in key areas ..."
www.canada.ca/en/innovatio...
www.canada.ca/en/innovatio...
Government of Canada launches new initiative to recruit world-leading researchers - Canada.ca
Canada will invest $1.7 billion to attract top global talent
www.canada.ca
December 10, 2025 at 12:24 AM
"Canada Impact+ Research Chairs program—a new $1 billion investment that will provide Canadian institutions the opportunity to recruit top-tier international researchers with expertise in key areas ..."
www.canada.ca/en/innovatio...
www.canada.ca/en/innovatio...
Reposted by Marlos C. Machado
Thrilled to start 2026 as faculty in Psych & CS
@ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
@ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
December 6, 2025 at 7:26 PM
Thrilled to start 2026 as faculty in Psych & CS
@ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
@ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
The Computing Science Dept. at the University of Alberta has multiple faculty job openings. Please share this broadly. We have a great environment!
- CS Theory: tinyurl.com/zrh9mk69
- Network/Cyber Security: tinyurl.com/renxazzy
- Robotics/CV/Graphics: tinyurl.com/ypcsfbff
- CS Theory: tinyurl.com/zrh9mk69
- Network/Cyber Security: tinyurl.com/renxazzy
- Robotics/CV/Graphics: tinyurl.com/ypcsfbff
November 27, 2025 at 6:00 PM
The Computing Science Dept. at the University of Alberta has multiple faculty job openings. Please share this broadly. We have a great environment!
- CS Theory: tinyurl.com/zrh9mk69
- Network/Cyber Security: tinyurl.com/renxazzy
- Robotics/CV/Graphics: tinyurl.com/ypcsfbff
- CS Theory: tinyurl.com/zrh9mk69
- Network/Cyber Security: tinyurl.com/renxazzy
- Robotics/CV/Graphics: tinyurl.com/ypcsfbff
The Department of Computing Science at the University of Alberta at the University of Alberta has an opening for another tenure-track faculty in robotics. Please, spread the word.
I can attest to how awesome our department and @amiithinks.bsky.social are!
(Official job posting coming soon.)
I can attest to how awesome our department and @amiithinks.bsky.social are!
(Official job posting coming soon.)
November 20, 2025 at 2:54 PM
The Department of Computing Science at the University of Alberta at the University of Alberta has an opening for another tenure-track faculty in robotics. Please, spread the word.
I can attest to how awesome our department and @amiithinks.bsky.social are!
(Official job posting coming soon.)
I can attest to how awesome our department and @amiithinks.bsky.social are!
(Official job posting coming soon.)
Ratatouille (2007)
October 7, 2025 at 9:58 PM
Ratatouille (2007)
This paper has now been accepted @neuripsconf.bsky.social !
Huge congratulations, Hon Tik (Rick) Tse and Siddarth Chandrasekar.
Huge congratulations, Hon Tik (Rick) Tse and Siddarth Chandrasekar.
📢 I'm happy to share the preprint: _Reward-Aware Proto-Representations in Reinforcement Learning_ ‼️
My PhD student, Hon Tik Tse, led this work, and my MSc student, Siddarth Chandrasekar, assisted us.
arxiv.org/abs/2505.16217
Basically, it's the SR with rewards. See below 👇
My PhD student, Hon Tik Tse, led this work, and my MSc student, Siddarth Chandrasekar, assisted us.
arxiv.org/abs/2505.16217
Basically, it's the SR with rewards. See below 👇
September 18, 2025 at 10:04 PM
This paper has now been accepted @neuripsconf.bsky.social !
Huge congratulations, Hon Tik (Rick) Tse and Siddarth Chandrasekar.
Huge congratulations, Hon Tik (Rick) Tse and Siddarth Chandrasekar.
2/2: “Conquerors live in dread of the day when they are shown to be, not superior, but simply lucky.”
― N.K. Jemisin, The Stone Sky
― N.K. Jemisin, The Stone Sky
August 27, 2025 at 2:20 AM
2/2: “Conquerors live in dread of the day when they are shown to be, not superior, but simply lucky.”
― N.K. Jemisin, The Stone Sky
― N.K. Jemisin, The Stone Sky
1/2: But there are none so frightened, or so strange in their fear, as conquerors. They conjure phantoms endlessly, terrified that their victims will someday do back what was done to them—even if, in truth, their victims couldn’t care less about such pettiness and have moved on.”
August 27, 2025 at 2:20 AM
1/2: But there are none so frightened, or so strange in their fear, as conquerors. They conjure phantoms endlessly, terrified that their victims will someday do back what was done to them—even if, in truth, their victims couldn’t care less about such pettiness and have moved on.”
Reposted by Marlos C. Machado
Excited to announce the RLC best paper awards! Like last year, we wanted to highlight the many excellent ways you can do research.
rl-conference.cc/RLC2025Award...
rl-conference.cc/RLC2025Award...
RLC 2025 - Outstanding Paper Awards
rl-conference.cc
August 7, 2025 at 8:30 PM
Excited to announce the RLC best paper awards! Like last year, we wanted to highlight the many excellent ways you can do research.
rl-conference.cc/RLC2025Award...
rl-conference.cc/RLC2025Award...
* RLC Journal to Conference Track:*
(Originally published at TMLR)
- Deep RL track (Thu): AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning by S. Pramanik
(Originally published at TMLR)
- Deep RL track (Thu): AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning by S. Pramanik
August 4, 2025 at 3:49 PM
* RLC Journal to Conference Track:*
(Originally published at TMLR)
- Deep RL track (Thu): AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning by S. Pramanik
(Originally published at TMLR)
- Deep RL track (Thu): AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning by S. Pramanik
* RLC Full Papers:*
(These are great papers!)
- Deep RL track (Thu): Deep Reinforcement Learning with Gradient Eligibility Traces by E. Elelimy
- Foundations track (Fri): An Analysis of Action-Value Temporal-Difference Methods That Learn State Values by B. Daley and P. Nagarajan
(These are great papers!)
- Deep RL track (Thu): Deep Reinforcement Learning with Gradient Eligibility Traces by E. Elelimy
- Foundations track (Fri): An Analysis of Action-Value Temporal-Difference Methods That Learn State Values by B. Daley and P. Nagarajan
August 4, 2025 at 3:49 PM
* RLC Full Papers:*
(These are great papers!)
- Deep RL track (Thu): Deep Reinforcement Learning with Gradient Eligibility Traces by E. Elelimy
- Foundations track (Fri): An Analysis of Action-Value Temporal-Difference Methods That Learn State Values by B. Daley and P. Nagarajan
(These are great papers!)
- Deep RL track (Thu): Deep Reinforcement Learning with Gradient Eligibility Traces by E. Elelimy
- Foundations track (Fri): An Analysis of Action-Value Temporal-Difference Methods That Learn State Values by B. Daley and P. Nagarajan
* RLC Workshop Papers (2/2):*
Inductive Biases in RL
sites.google.com/view/ibrl-wo...
- A Study of Value-Aware Eigenoptions by H. Kotamreddy
Inductive Biases in RL
sites.google.com/view/ibrl-wo...
- A Study of Value-Aware Eigenoptions by H. Kotamreddy
August 4, 2025 at 3:49 PM
* RLC Workshop Papers (2/2):*
Inductive Biases in RL
sites.google.com/view/ibrl-wo...
- A Study of Value-Aware Eigenoptions by H. Kotamreddy
Inductive Biases in RL
sites.google.com/view/ibrl-wo...
- A Study of Value-Aware Eigenoptions by H. Kotamreddy
* RLC Workshop Papers (1/2):*
RL Beyond Rewards
rlbrew2-workshop.github.io
- Tue 11:59 (spotlight talk): Towards An Option Basis To Optimize All Rewards by S. Chandrasekar
- The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis by A. Lewandowsi
RL Beyond Rewards
rlbrew2-workshop.github.io
- Tue 11:59 (spotlight talk): Towards An Option Basis To Optimize All Rewards by S. Chandrasekar
- The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis by A. Lewandowsi
Workshop on Reinforcement Learning Beyond Rewards: Ingredients for Developing Generalist Agents
rlbrew2-workshop.github.io
August 4, 2025 at 3:49 PM
* RLC Workshop Papers (1/2):*
RL Beyond Rewards
rlbrew2-workshop.github.io
- Tue 11:59 (spotlight talk): Towards An Option Basis To Optimize All Rewards by S. Chandrasekar
- The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis by A. Lewandowsi
RL Beyond Rewards
rlbrew2-workshop.github.io
- Tue 11:59 (spotlight talk): Towards An Option Basis To Optimize All Rewards by S. Chandrasekar
- The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis by A. Lewandowsi
Here's what our group will be presenting at RLC'25.
* Invited Talks at Workshops:*
Tue 10:00: The Causal RL Workshop sites.google.com/uci.edu/crlw...
Tue 14:30: Inductive Biases in RL (IBRL) Workshop
sites.google.com/view/ibrl-wo...
Tue 15:00: Panel Discussion at IBRL Workshop
* Invited Talks at Workshops:*
Tue 10:00: The Causal RL Workshop sites.google.com/uci.edu/crlw...
Tue 14:30: Inductive Biases in RL (IBRL) Workshop
sites.google.com/view/ibrl-wo...
Tue 15:00: Panel Discussion at IBRL Workshop
August 4, 2025 at 3:49 PM
Here's what our group will be presenting at RLC'25.
* Invited Talks at Workshops:*
Tue 10:00: The Causal RL Workshop sites.google.com/uci.edu/crlw...
Tue 14:30: Inductive Biases in RL (IBRL) Workshop
sites.google.com/view/ibrl-wo...
Tue 15:00: Panel Discussion at IBRL Workshop
* Invited Talks at Workshops:*
Tue 10:00: The Causal RL Workshop sites.google.com/uci.edu/crlw...
Tue 14:30: Inductive Biases in RL (IBRL) Workshop
sites.google.com/view/ibrl-wo...
Tue 15:00: Panel Discussion at IBRL Workshop
RLC starts tomorrow here in Edmonton. I couldn't be more excited! It has a fantastic roll of speakers, great papers, and workshops. And this time, it is in Edmonton 😁
@rl-conference.bsky.social is my favourite conference, and no, it is not because I am one of its organizers this year.
@rl-conference.bsky.social is my favourite conference, and no, it is not because I am one of its organizers this year.
August 4, 2025 at 3:27 PM
RLC starts tomorrow here in Edmonton. I couldn't be more excited! It has a fantastic roll of speakers, great papers, and workshops. And this time, it is in Edmonton 😁
@rl-conference.bsky.social is my favourite conference, and no, it is not because I am one of its organizers this year.
@rl-conference.bsky.social is my favourite conference, and no, it is not because I am one of its organizers this year.
This was a great long-term effort from @martinklissarov.bsky.social, Akhil Bagaria, and @ray-luo.bsky.social, and it led to a great overview of the ideas behind leveraging temporal abstractions in AI.
If anything, I think this is very useful resource for anyone interested in this field!
If anything, I think this is very useful resource for anyone interested in this field!
As AI agents face increasingly long and complex tasks, decomposing them into subtasks becomes increasingly appealing.
But how do we discover such temporal structure?
Hierarchical RL provides a natural formalism-yet many questions remain open.
Here's our overview of the field🧵
But how do we discover such temporal structure?
Hierarchical RL provides a natural formalism-yet many questions remain open.
Here's our overview of the field🧵
June 27, 2025 at 8:57 PM
This was a great long-term effort from @martinklissarov.bsky.social, Akhil Bagaria, and @ray-luo.bsky.social, and it led to a great overview of the ideas behind leveraging temporal abstractions in AI.
If anything, I think this is very useful resource for anyone interested in this field!
If anything, I think this is very useful resource for anyone interested in this field!
Reposted by Marlos C. Machado
To align better with workshop acceptance dates, 𝐑𝐋𝐂 𝐢𝐬 𝐞𝐱𝐭𝐞𝐧𝐝𝐢𝐧𝐠 𝐢𝐭𝐬 𝐞𝐚𝐫𝐥𝐲 𝐫𝐞𝐠𝐢𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐝𝐞𝐚𝐝𝐥𝐢𝐧𝐞 𝐭𝐨 𝐉𝐮𝐧𝐞 𝟐𝟑𝐫𝐝!
May 30, 2025 at 3:02 PM
To align better with workshop acceptance dates, 𝐑𝐋𝐂 𝐢𝐬 𝐞𝐱𝐭𝐞𝐧𝐝𝐢𝐧𝐠 𝐢𝐭𝐬 𝐞𝐚𝐫𝐥𝐲 𝐫𝐞𝐠𝐢𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐝𝐞𝐚𝐝𝐥𝐢𝐧𝐞 𝐭𝐨 𝐉𝐮𝐧𝐞 𝟐𝟑𝐫𝐝!
9/9: I genuinely think AgarCL might unlock new research avenues in CRL, including loss of plasticity, exploration, representation learning, and more. I do hope you consider using it.
Repo: github.com/machado-rese...
Website: agarcl.github.io
Preprint: arxiv.org/abs/2505.18347
Repo: github.com/machado-rese...
Website: agarcl.github.io
Preprint: arxiv.org/abs/2505.18347
May 27, 2025 at 3:48 AM
9/9: I genuinely think AgarCL might unlock new research avenues in CRL, including loss of plasticity, exploration, representation learning, and more. I do hope you consider using it.
Repo: github.com/machado-rese...
Website: agarcl.github.io
Preprint: arxiv.org/abs/2505.18347
Repo: github.com/machado-rese...
Website: agarcl.github.io
Preprint: arxiv.org/abs/2505.18347
8/9: Well, if you are still interested, you should probably consider reading the paper, but it is interesting to see that most of the agents we considered were able to reach human-level performance only in the most benign settings. And we did use a lot of computing here!
May 27, 2025 at 3:48 AM
8/9: Well, if you are still interested, you should probably consider reading the paper, but it is interesting to see that most of the agents we considered were able to reach human-level performance only in the most benign settings. And we did use a lot of computing here!
7/9: Through mini-games, we tried to quantify and isolate some of the challenges AgarCL poses, including partial observability, non-stationarity, exploration, hyperparameter tuning, and the non-episodic nature of the environment (so easy to forget!). Where do our agents "break"?
May 27, 2025 at 3:48 AM
7/9: Through mini-games, we tried to quantify and isolate some of the challenges AgarCL poses, including partial observability, non-stationarity, exploration, hyperparameter tuning, and the non-episodic nature of the environment (so easy to forget!). Where do our agents "break"?
6/9: Importantly, this is a challenge problem that forces us to deal with many problems we often avoid, such as hyperparameter sweeps and exploration in CRL.
It is perhaps no surprise that the classic algorithms we considered couldn't really make much progress in the full game.
It is perhaps no surprise that the classic algorithms we considered couldn't really make much progress in the full game.
May 27, 2025 at 3:48 AM
6/9: Importantly, this is a challenge problem that forces us to deal with many problems we often avoid, such as hyperparameter sweeps and exploration in CRL.
It is perhaps no surprise that the classic algorithms we considered couldn't really make much progress in the full game.
It is perhaps no surprise that the classic algorithms we considered couldn't really make much progress in the full game.
5/9: Over time, even the agent's observation will change, as the camera needs to zoom out to accommodate more agents; not to mention that there are other agents in the environment. I'm very excited about AgarCL because I think it allows us to ask questions we couldn't before.
May 27, 2025 at 3:48 AM
5/9: Over time, even the agent's observation will change, as the camera needs to zoom out to accommodate more agents; not to mention that there are other agents in the environment. I'm very excited about AgarCL because I think it allows us to ask questions we couldn't before.
4/9: AgarCL is an adaptation of agar.io, a game with simple mechanics that lead to complex interactions. It's non-episodic, and a key aspect is that the agent dynamics change as it accumulates mass: It becomes slower, gains new affordances, sheds more mass, etc.
May 27, 2025 at 3:48 AM
4/9: AgarCL is an adaptation of agar.io, a game with simple mechanics that lead to complex interactions. It's non-episodic, and a key aspect is that the agent dynamics change as it accumulates mass: It becomes slower, gains new affordances, sheds more mass, etc.
3/9: AgarCL is our attempt at an environment with the complexity of a "big world" but in a smooth way, where the "laws of physics" don't change. It has complex dynamics, is partially observable, with non-stationarity, pixel-based observations, and a hybrid action space.
May 27, 2025 at 3:48 AM
3/9: AgarCL is our attempt at an environment with the complexity of a "big world" but in a smooth way, where the "laws of physics" don't change. It has complex dynamics, is partially observable, with non-stationarity, pixel-based observations, and a hybrid action space.
2/9: CRL is often motivated by the idea that the world is bigger than the agent, requiring tracking. We usually simulate this with non-stationarity by cycling through classic episodic problems. I've written papers like this, but it feels too artificial.
arxiv.org/abs/2303.07507
arxiv.org/abs/2303.07507
May 27, 2025 at 3:48 AM
2/9: CRL is often motivated by the idea that the world is bigger than the agent, requiring tracking. We usually simulate this with non-stationarity by cycling through classic episodic problems. I've written papers like this, but it feels too artificial.
arxiv.org/abs/2303.07507
arxiv.org/abs/2303.07507