Mateusz Ostaszewski
@mateuszostaszewski.bsky.social
ML researcher @ Warsaw University of Technology
Reinforcement learning / Neural Networks Plasticity / Neural Network Representations / AI4Science
Reinforcement learning / Neural Networks Plasticity / Neural Network Representations / AI4Science
Reposted by Mateusz Ostaszewski
🚨Scaling RL
Most RL methods’ performance saturate at ~5 layers. In this work led by Kevin Wang, we crack the right configuration for scaling Contrastive RL and go beyond 1000 layers NNs! Deep NNs unlock emergent behaviors and other cool properties. Check out Kevin’s thread!
Most RL methods’ performance saturate at ~5 layers. In this work led by Kevin Wang, we crack the right configuration for scaling Contrastive RL and go beyond 1000 layers NNs! Deep NNs unlock emergent behaviors and other cool properties. Check out Kevin’s thread!
1/ While most RL methods use shallow MLPs (~2–5 layers), we show that scaling up to 1000-layers for contrastive RL (CRL) can significantly boost performance, ranging from doubling performance to 50x on a diverse suite of robotic tasks.
Webpage+Paper+Code: wang-kevin3290.github.io/scaling-crl/
Webpage+Paper+Code: wang-kevin3290.github.io/scaling-crl/
March 20, 2025 at 11:22 PM
🚨Scaling RL
Most RL methods’ performance saturate at ~5 layers. In this work led by Kevin Wang, we crack the right configuration for scaling Contrastive RL and go beyond 1000 layers NNs! Deep NNs unlock emergent behaviors and other cool properties. Check out Kevin’s thread!
Most RL methods’ performance saturate at ~5 layers. In this work led by Kevin Wang, we crack the right configuration for scaling Contrastive RL and go beyond 1000 layers NNs! Deep NNs unlock emergent behaviors and other cool properties. Check out Kevin’s thread!
Reposted by Mateusz Ostaszewski
🚀 What happens when you modify the spectrum of singular values of the merged task vector? 🤔
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
February 10, 2025 at 2:47 PM
🚀 What happens when you modify the spectrum of singular values of the merged task vector? 🤔
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Reposted by Mateusz Ostaszewski
😊 Happy to Share!
🎉 Our paper "Learning Graph Representation of Agent Diffusers (LGR-AD)" has been accepted as a full paper at #AAMAS (A*) International Conference on Autonomous Agents and Multiagent Systems!
#diffusion #graphs #agentsystem
@ideas-ncbr.bsky.social #WarszawUniversityOfTechnology
🎉 Our paper "Learning Graph Representation of Agent Diffusers (LGR-AD)" has been accepted as a full paper at #AAMAS (A*) International Conference on Autonomous Agents and Multiagent Systems!
#diffusion #graphs #agentsystem
@ideas-ncbr.bsky.social #WarszawUniversityOfTechnology
December 20, 2024 at 3:27 PM
😊 Happy to Share!
🎉 Our paper "Learning Graph Representation of Agent Diffusers (LGR-AD)" has been accepted as a full paper at #AAMAS (A*) International Conference on Autonomous Agents and Multiagent Systems!
#diffusion #graphs #agentsystem
@ideas-ncbr.bsky.social #WarszawUniversityOfTechnology
🎉 Our paper "Learning Graph Representation of Agent Diffusers (LGR-AD)" has been accepted as a full paper at #AAMAS (A*) International Conference on Autonomous Agents and Multiagent Systems!
#diffusion #graphs #agentsystem
@ideas-ncbr.bsky.social #WarszawUniversityOfTechnology
Excited to invite you to our #NeurIPS spotlight poster "BRO: Bigger, Regularized, Optimistic"! 🎉
📍 Poster #6302
📅 West Ballroom A-D
🕚 Friday, 11:00-14:00
Join us to discuss with Michał Nauman and me. Let’s talk SOTA in RL! 💪
🧵👇
📍 Poster #6302
📅 West Ballroom A-D
🕚 Friday, 11:00-14:00
Join us to discuss with Michał Nauman and me. Let’s talk SOTA in RL! 💪
🧵👇
December 12, 2024 at 3:58 PM
Excited to invite you to our #NeurIPS spotlight poster "BRO: Bigger, Regularized, Optimistic"! 🎉
📍 Poster #6302
📅 West Ballroom A-D
🕚 Friday, 11:00-14:00
Join us to discuss with Michał Nauman and me. Let’s talk SOTA in RL! 💪
🧵👇
📍 Poster #6302
📅 West Ballroom A-D
🕚 Friday, 11:00-14:00
Join us to discuss with Michał Nauman and me. Let’s talk SOTA in RL! 💪
🧵👇
Reposted by Mateusz Ostaszewski
Self-supervised Learning with Masked Autoencoders (MAE) is known to produce worse image representations than Joint-Embedding approaches (e.g. DINO). In our new paper, we identify new reasons for why that is and point towards solutions: arxiv.org/abs/2412.03215 🧵
December 5, 2024 at 7:56 PM
Self-supervised Learning with Masked Autoencoders (MAE) is known to produce worse image representations than Joint-Embedding approaches (e.g. DINO). In our new paper, we identify new reasons for why that is and point towards solutions: arxiv.org/abs/2412.03215 🧵
Reposted by Mateusz Ostaszewski
My first post here :)
Excellent news from NeurIPS. Two papers in, including a spotlight.
1. Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
2. Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
Excellent news from NeurIPS. Two papers in, including a spotlight.
1. Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
2. Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
November 26, 2024 at 12:27 PM
My first post here :)
Excellent news from NeurIPS. Two papers in, including a spotlight.
1. Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
2. Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
Excellent news from NeurIPS. Two papers in, including a spotlight.
1. Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
2. Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
Reposted by Mateusz Ostaszewski
💠#NeurIPS2024 is coming soon!
📜Let's focus on the BRO algorithm, introduced in 🔦spotlight paper by Michał Nauman, @mateuszostaszewski.bsky.social, Krzysztof Jankowski, @piotrmilos.bsky.social, Marek Cygan, to find out why the second dog runs better ➡️ ideas-ncbr.pl/en/bro-algor...
📜Let's focus on the BRO algorithm, introduced in 🔦spotlight paper by Michał Nauman, @mateuszostaszewski.bsky.social, Krzysztof Jankowski, @piotrmilos.bsky.social, Marek Cygan, to find out why the second dog runs better ➡️ ideas-ncbr.pl/en/bro-algor...
December 6, 2024 at 11:11 AM
💠#NeurIPS2024 is coming soon!
📜Let's focus on the BRO algorithm, introduced in 🔦spotlight paper by Michał Nauman, @mateuszostaszewski.bsky.social, Krzysztof Jankowski, @piotrmilos.bsky.social, Marek Cygan, to find out why the second dog runs better ➡️ ideas-ncbr.pl/en/bro-algor...
📜Let's focus on the BRO algorithm, introduced in 🔦spotlight paper by Michał Nauman, @mateuszostaszewski.bsky.social, Krzysztof Jankowski, @piotrmilos.bsky.social, Marek Cygan, to find out why the second dog runs better ➡️ ideas-ncbr.pl/en/bro-algor...