Hi @raspstephan.bsky.social - interesting graph! However, I am not sure it makes sense to plot the ensemble and deterministic comparisons on the same axes. I guess the AIFS-CRPS results are from Fig9, which compares against the Tco1279 ENS rather than HRES…?

More details on fair ensemble scores:

rmets.onlinelibrary.wiley.com/doi/abs/10.1...

rmets.onlinelibrary.wiley.com/doi/full/10....

This is because optimising a fair ensemble score means there is no tension between minimising error and maintaining realistic levels of variability, which is unavoidable for deterministic MSE training. fCRPS loss is minimised when the forecast is drawn from the same distribution as observations.

Hmm - maybe Bluesky doesn’t support self-uploaded GIFs yet? Sorry Simon!

… Peter Deuben , Sara Hahner, Pedro Maciel, Ana Prieto-Nemesio, Cathal O'Brien, Florian Pinault, Jan Polster, Baudouin Raoult, Steffen Tietsche, Martin Leutbecher!

This work represents a huge team effort from @ECMWF colleagues Simon Lang, Mihai Alexe, Mariana Clare, Rilwan Adewoyin, Zied Ben Bouallègue, Matthew Chantry, Jesper Dramsch…

For subseasonal forecasts, AIFS-CRPS outperforms the IFS ensemble before calibration and is competitive with the IFS ensemble when forecasts are evaluated as anomalies to remove the influence of model biases.

For medium-range forecasts, AIFS-CRPS outperforms the physics-based Integrated Forecasting System (IFS) ensemble for the majority of variables and lead times.

The trained model is stochastic and can generate as many exchangeable members as desired and is computationally feasible in inference.

Specifically, it introduces the “almost fair CRPS”, which approximately removes the bias in the score due to finite ensemble size yet avoids a degeneracy of the fair CRPS.

AIFS-CRPS is an ensemble variant of the Artificial Intelligence Forecasting System (AIFS) developed at ECMWF. The training protocol utilises a probabilistic loss function based on the Continuous Ranked Probability Score (CRPS).

This paper presents a novel approach to obtain a weather forecast model for ensemble forecasting with machine-learning.

AIFS-CRPS is a variant of the Artificial Intelligence Forecasting System (AIFS) developed at
ECMWF. Its loss function is based on a proper score, the Continuous Ranked Probability Score (CRPS).

This paper presents a novel approach to obtain a weather forecast model for ensemble forecasting with machine-learning.