it's so close to being awesome 😭 but skimming code... ligand = any heteroatom (rip waters, ions, buffers). assumes atoms in prediction are perfectly ordered/labeled - not true for chai, boltz, etc. binding site = assumes identical residue IDs.
shameless promo, we hit most of these in np & np-bench ❤️

this is so incredibly sweet!!! happy hanukkah ♥️ making my latkes later today and i’ll be sure to grate the onion first ;)

it rigorously and consistently evaluates traditional structure prediction metrics like pocket-aligned ligand rmsd and dockq score. we hope this helps create more consistency in how we evaluate progress in the field of structure prediction & ultimately aids in pushing the field forward as a whole :)

also, ConfBench is just one of many aspects of the NP3 technical report - we are also excited to announce we're open-sourcing NPBench: our turnkey dataset distillation and code library for standardized structure prediction benchmarking!

unimaginably large thank you to the amazing team and leadership that made this work possible, especially Zhuoran Qiao and Matt Welborn for believing in my vision of conformational enablement of structure prediction!

& to @rommieamaro.bsky.social, who sparked my love for dynamic proteins ♡