Stephan Rasp
@raspstephan.bsky.social
Environmental science
39%
Geology
15%
Stephan Rasp
@raspstephan.bsky.social
· Feb 13
Other minor updates:
- Where available, we added 2022 as an eval year in the interactive graphics.
- We added forecast activity as a metric for deterministic models, a simple measure of blurring.
- More regions.
Don't hesitate to file bugs or suggestions as GitHub issues.
end/
- Where available, we added 2022 as an eval year in the interactive graphics.
- We added forecast activity as a metric for deterministic models, a simple measure of blurring.
- More regions.
Don't hesitate to file bugs or suggestions as GitHub issues.
end/
Stephan Rasp
@raspstephan.bsky.social
· Feb 13
Next, we added 4 new models to the public benchmark (which now also uses WB-X as a backend):
- GenCast
- Stormer
- Excarta (HEAL-ViT)
- ArchesWeather
The probabilistic scorecard finally looks a little more populated :)
4/
- GenCast
- Stormer
- Excarta (HEAL-ViT)
- ArchesWeather
The probabilistic scorecard finally looks a little more populated :)
4/
Stephan Rasp
@raspstephan.bsky.social
· Feb 13
WeatherBench-X documentationContentsMenuExpandLight modeDark modeAuto light/dark mode
weatherbench-x.readthedocs.io
To get started, check out the documentation: weatherbench-x.readthedocs.io/en/latest/
For an example of evaluating forecasts against sparse obs, see: weatherbench-x.readthedocs.io/en/latest/ho...
Please don't hesitate to ask questions or report bugs/feature requests via a GitHub issue :)
3/n
For an example of evaluating forecasts against sparse obs, see: weatherbench-x.readthedocs.io/en/latest/ho...
Please don't hesitate to ask questions or report bugs/feature requests via a GitHub issue :)
3/n
Stephan Rasp
@raspstephan.bsky.social
· Feb 13
WB-X is a complete rewrite of our evaluation code. We designed it to be as modular and powerful as possible with cutting-edge use cases like observation-based models in mind. We've used WB-X internally over the last year for most of our model development.
2/n
2/n
Stephan Rasp
@raspstephan.bsky.social
· Feb 13
GitHub - google-research/weatherbenchX: A modular framework for evaluating weather forecasts
A modular framework for evaluating weather forecasts - google-research/weatherbenchX
github.com
🚨 WeatherBench Update
1. WeatherBench-X, our new evaluation code, is now on GitHub: github.com/google-resea...
2. New models (plus other small updates) on the WeatherBench website: sites.research.google/weatherbench/
1/n
1. WeatherBench-X, our new evaluation code, is now on GitHub: github.com/google-resea...
2. New models (plus other small updates) on the WeatherBench website: sites.research.google/weatherbench/
1/n
Reposted by Stephan Rasp
Randy Chase
@dopplerchase.bsky.social
· Dec 31
2025 is here tomorrow, so let's reflect on 2024. Even without the final counts and the new AMS and AGU ML journals, 2024 has eclipsed 10% of all papers and had over 600 papers mentioning neural networks in their abstracts 📈
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
Deterministic scores – WeatherBench2
sites.research.google
Sure. The y-axis shows the 3d T850 RMSE relative to ECMWF IFS HRES (so >100% = better). It's a crude attempt at normalizing different evaluations, so don't overinterpret the small differences. This is more about the bigger picture.
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
So, for AIFS and GenCast I am evaluating the ensemble mean. I still use deterministic HRES as a reference. For AIFS I grabbed the NH HRES scores from the scorecard on the ECMWF website and then eyeballed the AIFS score from Fig 9.
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
AI-Weather SotA vs Time
The purpose of this spreadsheet is not to exactly compare different models but rather to get an overall sense of progress in AI-based weather prediction.
doi.org
Good idea, done: Rasp, Stephan (2024). AI-Weather SotA vs Time. figshare. Dataset. doi.org/10.6084/m9.f...
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
But you do raise a good point. for purely obs-trained models, this probably isn't a fair comparison. In this case the conclusions are probably the same but still.
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
True but in the medium-range the obs uncertainty is probably smaller than the forecast uncertainty, right? Radiosonde vs ERA5 RMSE ~ 1k, right?
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
What is the conclusion from GraphDOP being so far away from SotA? Is the setup still suboptimal in some way or is pure obs-based forecasting harder than some might have thought.
Stephan Rasp
@raspstephan.bsky.social
· Dec 23
ECMWF with two new papers right before christmas.
AIFS-CRPS: arxiv.org/abs/2412.158...
GraphDOP (the first truly end2end global weather model): arxiv.org/abs/2412.15687
Here they are added to the SotA tracker: docs.google.com/spreadsheets...
AIFS-CRPS: arxiv.org/abs/2412.158...
GraphDOP (the first truly end2end global weather model): arxiv.org/abs/2412.15687
Here they are added to the SotA tracker: docs.google.com/spreadsheets...
Reposted by Stephan Rasp
Stephan Hoyer
@stephanhoyer.com
· Dec 19
Neural general circulation models optimized to predict satellite-based precipitation observations
Climate models struggle to accurately simulate precipitation, particularly extremes and the diurnal cycle. Here, we present a hybrid model that is trained directly on satellite-based precipitation obs...
arxiv.org
Can incorporating AI improve precipitation in global weather and climate models?
Yes! In the latest NeuralGCM paper, we show that training on satellite-based precipitation results in significant improvements over traditional atmospheric models:
arxiv.org/abs/2412.11973
Yes! In the latest NeuralGCM paper, we show that training on satellite-based precipitation results in significant improvements over traditional atmospheric models:
arxiv.org/abs/2412.11973
Reposted by Stephan Rasp
🌎
Stephan Rasp
@raspstephan.bsky.social
· Nov 19
👋