8/13 Many perturbation prediction evaluations use 2,000 HVGs, while most genes don't show a strong response. However, even when narrowing down to the top 20 DEGs per perturbation, some scFM embeddings only slightly outperformed the baseline methods, while others still didn't.

7/13 We found that current-generation zero-shot scFM embeddings showed no significant improvement over task-specific model GEARS or even over simple baselines when predicting perturbation effects across 2,000 highly variable genes (HVGs).

1/13 Excited to share that PertEval-scFM got accepted into ICML 2025 🇨🇦!

We provide benchmark and evaluation tools for perturbation effect prediction models, including single-cell foundation models (scFMs). Paper and GitHub link at the end of the thread! 🧵👇

Thank you for your great work. We have a copy here at the office proudly on display 😎

Attended an incredible talk by @philipcball.bsky.social in Oxford. He covered a big chunk of genetics and molecular biology at lightning speed without compromising on clarity. He also convincingly explained why the central dogma is outdated. Go check out his latest book, How Life Works