KS
@kazzorr.bsky.social
Reposted by KS
Rapid RL experimentation is great. But how do you catch silent errors before they slip by?
In this post, I share tools and habits that help me move quickly from idea to result without sacrificing reliability.
In this post, I share tools and habits that help me move quickly from idea to result without sacrificing reliability.
How to catch subtle RL bugs before they catch you
Tools and habits for reliable, fast RL experimentation and development
open.substack.com
October 13, 2025 at 11:29 AM
Rapid RL experimentation is great. But how do you catch silent errors before they slip by?
In this post, I share tools and habits that help me move quickly from idea to result without sacrificing reliability.
In this post, I share tools and habits that help me move quickly from idea to result without sacrificing reliability.
Reposted by KS
If you're training models with more than one loss term, I can again strongly recommend our ConFIG optimizer: tum-pbs.github.io/ConFIG/ , simply swap out Adam&Co. for ConFIG, and you can potentially see substantial reductions in your training loss 😁 We'd also be curious to hear how it works for you
December 10, 2024 at 7:11 AM
If you're training models with more than one loss term, I can again strongly recommend our ConFIG optimizer: tum-pbs.github.io/ConFIG/ , simply swap out Adam&Co. for ConFIG, and you can potentially see substantial reductions in your training loss 😁 We'd also be curious to hear how it works for you
Reposted by KS
The densest water in the world ocean forms around Antarctica, and then spreads out to flood the global abyss. We’ve seen the production of this water dwindle over the past few decades, a change that is now impacting the North Atlantic. 🌊🧪❄️
www.nature.com/articles/s41...
www.nature.com/articles/s41...
Weakening of the Atlantic Meridional Overturning Circulation abyssal limb in the North Atlantic - Nature Geoscience
Mooring observations and hydrographic data suggest the Atlantic Meridional Overturning Circulation abyssal limb has weakened over the past two decades in the North Atlantic, most likely due to reduced...
www.nature.com
January 4, 2025 at 9:45 AM
The densest water in the world ocean forms around Antarctica, and then spreads out to flood the global abyss. We’ve seen the production of this water dwindle over the past few decades, a change that is now impacting the North Atlantic. 🌊🧪❄️
www.nature.com/articles/s41...
www.nature.com/articles/s41...
Reposted by KS
We have a new paper out in @science.org today, led by Helge Goessling from #AWI:
The recent global temperature surge in 2023 was intensified by a record-low planetary #albedo
👉 doi.org/10.1126/scie... @thomasjung.bsky.social @ecmwf.bsky.social
This is what we found (🧵1/8)
The recent global temperature surge in 2023 was intensified by a record-low planetary #albedo
👉 doi.org/10.1126/scie... @thomasjung.bsky.social @ecmwf.bsky.social
This is what we found (🧵1/8)
Recent global temperature surge intensified by record-low planetary albedo
In 2023, the global mean temperature soared to almost 1.5K above the pre-industrial level, surpassing the previous record by about 0.17K. Previous best-guess estimates of known drivers including anthr...
doi.org
December 5, 2024 at 7:18 PM
We have a new paper out in @science.org today, led by Helge Goessling from #AWI:
The recent global temperature surge in 2023 was intensified by a record-low planetary #albedo
👉 doi.org/10.1126/scie... @thomasjung.bsky.social @ecmwf.bsky.social
This is what we found (🧵1/8)
The recent global temperature surge in 2023 was intensified by a record-low planetary #albedo
👉 doi.org/10.1126/scie... @thomasjung.bsky.social @ecmwf.bsky.social
This is what we found (🧵1/8)
Reposted by KS
Okay, I have to do a thread on this pretty amazing article because it is just seriously chockers with eye-opening stats about 'direct air capture' - ie, sucking up air and trying to remove carbon dioxide from it
news.mit.edu/2024/reality...
news.mit.edu/2024/reality...
Reality check on technologies to remove carbon dioxide from the air
An MIT Energy Initiative study finds many climate-stabilization plans are based on questionable assumptions about the future cost and deployment of “direct air capture” and therefore may not bring abo...
news.mit.edu
November 26, 2024 at 8:26 AM
Okay, I have to do a thread on this pretty amazing article because it is just seriously chockers with eye-opening stats about 'direct air capture' - ie, sucking up air and trying to remove carbon dioxide from it
news.mit.edu/2024/reality...
news.mit.edu/2024/reality...
Reposted by KS
Graph Transformers (GTs) can handle long-range dependencies and resolve information bottlenecks, but they’re computationally expensive. Our new model, Spexphormer, helps scale them to much larger graphs – check it out at NeurIPS next week, or the preview here!
[1/13]
#NeurIPS2024
[1/13]
#NeurIPS2024
December 5, 2024 at 7:58 PM
Graph Transformers (GTs) can handle long-range dependencies and resolve information bottlenecks, but they’re computationally expensive. Our new model, Spexphormer, helps scale them to much larger graphs – check it out at NeurIPS next week, or the preview here!
[1/13]
#NeurIPS2024
[1/13]
#NeurIPS2024
Reposted by KS
A common question nowadays: Which is better, diffusion or flow matching? 🤔
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
December 2, 2024 at 6:45 PM
A common question nowadays: Which is better, diffusion or flow matching? 🤔
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
Reposted by KS
🧵 Today with @polymathicai.bsky.social and others we're releasing two massive datasets that span dozens of fields - from bacterial growth to supernova!
We want this to enable multi-disciplinary foundation model research.
We want this to enable multi-disciplinary foundation model research.
December 2, 2024 at 7:46 PM
🧵 Today with @polymathicai.bsky.social and others we're releasing two massive datasets that span dozens of fields - from bacterial growth to supernova!
We want this to enable multi-disciplinary foundation model research.
We want this to enable multi-disciplinary foundation model research.
Reposted by KS
'Global emergence of regional heatwave hotspots outpaces climate model simulations'
our new paper in @pnas.org with S. Bartusek, R. Seager. J. Schellnhuber and M. Ting investigating the tail behaviour of extreme heatwave trends.
@iiasa.ac.at @columbiaclimate.bsky.social @lamontearth.bsky.social
our new paper in @pnas.org with S. Bartusek, R. Seager. J. Schellnhuber and M. Ting investigating the tail behaviour of extreme heatwave trends.
@iiasa.ac.at @columbiaclimate.bsky.social @lamontearth.bsky.social
November 26, 2024 at 6:51 PM
'Global emergence of regional heatwave hotspots outpaces climate model simulations'
our new paper in @pnas.org with S. Bartusek, R. Seager. J. Schellnhuber and M. Ting investigating the tail behaviour of extreme heatwave trends.
@iiasa.ac.at @columbiaclimate.bsky.social @lamontearth.bsky.social
our new paper in @pnas.org with S. Bartusek, R. Seager. J. Schellnhuber and M. Ting investigating the tail behaviour of extreme heatwave trends.
@iiasa.ac.at @columbiaclimate.bsky.social @lamontearth.bsky.social
Reposted by KS
I'm excited to share our APEBench paper arxiv.org/abs/2411.00180 and code github.com/tum-pbs/apeb..., to be presented at #NeurIPS. Congratulations Felix and Simon 😀 👍 At its core, APEBench features a lightning-fast ⚡️ fully differentiable spectral solver with a huge range of different PDEs.
November 27, 2024 at 2:18 AM
I'm excited to share our APEBench paper arxiv.org/abs/2411.00180 and code github.com/tum-pbs/apeb..., to be presented at #NeurIPS. Congratulations Felix and Simon 😀 👍 At its core, APEBench features a lightning-fast ⚡️ fully differentiable spectral solver with a huge range of different PDEs.
Reposted by KS
If you think Direct Air Capture (DAC) is going to save us, please read this.
This is isn’t some left-wing fringe environmental group saying this.
It’s MIT.
news.mit.edu/2024/reality...
This is isn’t some left-wing fringe environmental group saying this.
It’s MIT.
news.mit.edu/2024/reality...
Reality check on technologies to remove carbon dioxide from the air
An MIT Energy Initiative study finds many climate-stabilization plans are based on questionable assumptions about the future cost and deployment of “direct air capture” and therefore may not bring abo...
news.mit.edu
November 26, 2024 at 2:56 PM
If you think Direct Air Capture (DAC) is going to save us, please read this.
This is isn’t some left-wing fringe environmental group saying this.
It’s MIT.
news.mit.edu/2024/reality...
This is isn’t some left-wing fringe environmental group saying this.
It’s MIT.
news.mit.edu/2024/reality...
Reposted by KS
I am happy to share here our paper: "Spontaneous symmetry breaking in generative diffusion models", published at Neurips 2023.
We found that the generative capabilities of diffusion models are the result of a phase transition!
Preprint: arxiv.org/abs/2305.19693
Code: github.com/gabrielraya/...
We found that the generative capabilities of diffusion models are the result of a phase transition!
Preprint: arxiv.org/abs/2305.19693
Code: github.com/gabrielraya/...
November 22, 2024 at 5:48 PM
I am happy to share here our paper: "Spontaneous symmetry breaking in generative diffusion models", published at Neurips 2023.
We found that the generative capabilities of diffusion models are the result of a phase transition!
Preprint: arxiv.org/abs/2305.19693
Code: github.com/gabrielraya/...
We found that the generative capabilities of diffusion models are the result of a phase transition!
Preprint: arxiv.org/abs/2305.19693
Code: github.com/gabrielraya/...
Reposted by KS
Since I'm moving over, I would like to reshare my most recent blog post on why you should learn Optimal Transport. Part 2 is coming, most likely during the winter break :)
mufan-li.github.io/OT1/
mufan-li.github.io/OT1/
Everyone Should Learn Optimal Transport, Part 1
In my opinion, optimal transport (OT) is a seriously underrated topic. I think part of the reason is the way OT is often introduced: as an optimization problem or a metric on probability distributions...
mufan-li.github.io
November 22, 2024 at 7:58 PM
Since I'm moving over, I would like to reshare my most recent blog post on why you should learn Optimal Transport. Part 2 is coming, most likely during the winter break :)
mufan-li.github.io/OT1/
mufan-li.github.io/OT1/
Reposted by KS
Even as an interpretable ML researcher, I wasn't sure what to make of Mechanistic Interpretability, which seemed to come out of nowhere not too long ago.
But then I found the paper "Mechanistic?" by
@nsaphra.bsky.social and @sarah-nlp.bsky.social, which clarified things.
But then I found the paper "Mechanistic?" by
@nsaphra.bsky.social and @sarah-nlp.bsky.social, which clarified things.
November 20, 2024 at 8:00 AM
Even as an interpretable ML researcher, I wasn't sure what to make of Mechanistic Interpretability, which seemed to come out of nowhere not too long ago.
But then I found the paper "Mechanistic?" by
@nsaphra.bsky.social and @sarah-nlp.bsky.social, which clarified things.
But then I found the paper "Mechanistic?" by
@nsaphra.bsky.social and @sarah-nlp.bsky.social, which clarified things.
Reposted by KS
Thrilled to announce Boltz-1, the first open-source and commercially available model to achieve AlphaFold3-level accuracy on biomolecular structure prediction! An exciting collaboration with Jeremy, Saro, and an amazing team at MIT and Genesis Therapeutics. A thread!
November 17, 2024 at 4:20 PM
Thrilled to announce Boltz-1, the first open-source and commercially available model to achieve AlphaFold3-level accuracy on biomolecular structure prediction! An exciting collaboration with Jeremy, Saro, and an amazing team at MIT and Genesis Therapeutics. A thread!
Reposted by KS
Important new study shows that current climate models underestimate the human-caused slowing of the #AMOC (Atlantic Meridional Overturning Circulation), because they neglect freshwater influx from Greenland melt and other sources. /1 🌊
www.nature.com/articles/s41...
www.nature.com/articles/s41...
Weakening of the Atlantic Meridional Overturning Circulation driven by subarctic freshening since the mid-twentieth century - Nature Geoscience
Fresh meltwater entering the Labrador and Irminger seas has resulted in a slowing of the Atlantic Meridional Overturning Circulation since the 1950s, according to a combination of modelling approaches...
www.nature.com
November 20, 2024 at 11:22 AM
Important new study shows that current climate models underestimate the human-caused slowing of the #AMOC (Atlantic Meridional Overturning Circulation), because they neglect freshwater influx from Greenland melt and other sources. /1 🌊
www.nature.com/articles/s41...
www.nature.com/articles/s41...
Reposted by KS
🍏 New preprint alert! 🍏
PoM: Efficient Image and Video Generation with the Polynomial Mixer
arxiv.org/abs/2411.12663
This is my latest "summer project" and it was so big I had to call in reinforcements (Thanks @nicolasdufour.bsky.social)
TL;DR Transformers are for boomers, welcome to the future
🧵👇
PoM: Efficient Image and Video Generation with the Polynomial Mixer
arxiv.org/abs/2411.12663
This is my latest "summer project" and it was so big I had to call in reinforcements (Thanks @nicolasdufour.bsky.social)
TL;DR Transformers are for boomers, welcome to the future
🧵👇
PoM: Efficient Image and Video Generation with the Polynomial Mixer
Diffusion models based on Multi-Head Attention (MHA) have become ubiquitous to generate high quality images and videos. However, encoding an image or a video as a sequence of patches results in costly...
arxiv.org
November 20, 2024 at 8:08 AM
🍏 New preprint alert! 🍏
PoM: Efficient Image and Video Generation with the Polynomial Mixer
arxiv.org/abs/2411.12663
This is my latest "summer project" and it was so big I had to call in reinforcements (Thanks @nicolasdufour.bsky.social)
TL;DR Transformers are for boomers, welcome to the future
🧵👇
PoM: Efficient Image and Video Generation with the Polynomial Mixer
arxiv.org/abs/2411.12663
This is my latest "summer project" and it was so big I had to call in reinforcements (Thanks @nicolasdufour.bsky.social)
TL;DR Transformers are for boomers, welcome to the future
🧵👇
Reposted by KS
Interested in machine learning in science?
Timo and I recently published a book, and even if you are not a scientist, you'll find useful overviews of topics like causality and robustness.
The best part is that you can read it for free: ml-science-book.com
Timo and I recently published a book, and even if you are not a scientist, you'll find useful overviews of topics like causality and robustness.
The best part is that you can read it for free: ml-science-book.com
November 15, 2024 at 9:46 AM
Interested in machine learning in science?
Timo and I recently published a book, and even if you are not a scientist, you'll find useful overviews of topics like causality and robustness.
The best part is that you can read it for free: ml-science-book.com
Timo and I recently published a book, and even if you are not a scientist, you'll find useful overviews of topics like causality and robustness.
The best part is that you can read it for free: ml-science-book.com