Yep, that’s a cool idea. Should be quite easy to try actually

Afaik, not really. Recycling is done before diffusion module and the final output coords are not used for a refinement. Someone correct me if I am wrong though. But seems top method in this challenge could done smth like that: polarishub.io/competitions...

That being said, today I looked at the templates that AF3 used and actually none of the predictions used ground-truth template or templates after the training cutoff, so was a false alarm luckily. Would still be interesting to check why some predictions improved in the lowest bins.

It's not expected AF3 models would have perfect fidelity when run with ground-truth templates, especially cause it uses multiple templates and also MSA features. Additionally, only protein part is used from the templates, not the ligand information.

I want to thank my co-authors @jeeberhardt.bsky.social, @torstenschwede.bsky.social, @ninjani.bsky.social and all of our collaborators! RunsN’ Poses builds on PLINDER and OpenStructure—this work wouldn’t be possible without them!
Also thanks to @rokbreznikar.bsky.social for this amazing logo! 9/9

We’re pre-printing this early to get feedback from the community. We teamed up with @caswognum.nl at @polarishub.io to make the dataset and benchmark ML-ready polarishub.io/datasets/pli.... See also our github.com/plinder-org/... for more. Would love to hear your thoughts! (8/n

Our findings highlight the need for specialised benchmarks for deep learning in protein-ligand interaction prediction. As tasks like co-folding grow more complex, new metrics are needed to assess leakage and difficulty-best defined by interaction similarity, not protein or ligand similarity. (7/n)

To separate pose prediction from ranking, we analyzed all 25 models per method. Top-ranked models (black) outperform the worst (red) but still lag behind the best possible (blue). Success rates still correlate with training similarity, highlighting a key limitation of these methods. (6/n)

The largest cluster has 171 SARS-CoV-2 MPro X-ray structures with different small molecules. Despite explicit chirality in the input, all methods misoriented the 5-chlorobenzofuran-3-aminomethyl group, perhaps mimicking the 5-chloro-2-methoxyphenyl group from the closest training system. (5/n)

To ensure the observed trend is not driven by an over-representation of certain protein families, we also analyzed only cluster representatives to account for potential biases. The observed correlation doesn’t change. (4/n)

Interestingly, we found that ligands with abundant data in PDB, like cofactors, show better prediction performance. When excluding them to focus on drug-like ligands, the correlation between success rate and training similarity becomes even more linear. (3/n)

In this work, we explored how training data similarity impacts protein-ligand prediction accuracy—an overlooked aspect in recent benchmarks. Our analysis shows that the current co-folding methods struggle to generalize beyond ligand poses in their training data.(2/n)