Hello world! It's been a true joy to work on this with Michael. Both him and cryoJAX are things to keep an eye on. They're going to make amazing things happen in cryo and beyond.

The last metric we used calculates the likelihood that a given ensemble of volumes is contained in a set of images. This one is really exciting because we don't need any ground truth to compare to. We see the same trends between the simulated and experimental datasets.

The next is based on optimal transport. By calculating the differences between maps in a submission and the ground truth, we can calculate how close a submission is to the truth. We can also see how far from optimal participants estimated their populations.

Now comes the real work. Comparing all this amazing data! We came up with three novel ways of comparing series of volumes. With the first we simply perform PCA on the voxel maps and compare their bases. Turns out to be a super useful way of comparing motions between submissions.

We had amazing participation. Almost every method in the literature submitted to this challenge. We've chosen to keep the identity of the submissions anonymous (you'll see them in the manuscript denoted by icecream flavor hehe).

That's why we launched this challenge. We made two datasets, one experimental and one simulated. The simulated one was made with structures from molecular dynamics simulations. Here we hid a triple Gaussian for participants to discover. (Movie credit David Herreros).

There has been an explosion of methods to determine molecular ensembles from single particle cryo-EM datasets. However, these methods all use totally different algorithms and it's currently unknown how to assess their accuracy without ground truth structures to compare with.