NSF-Simons AI Institute for Cosmic Origins (CosmicAI)
@nsfsimonscosmicai.bsky.social
Funded by the U.S. National Science Foundation and the Simons Foundation. Driving AI innovation to explore cosmic origins and make astronomy more accessible and collaborative.
September 25, 2025 at 7:17 PM
Learn more
Website astrovisbench.github.io
Paper arxiv.org/abs/2505.20538
@utaustin.bsky.social @simonsfoundation.org @uvapress.bsky.social @nyupress.bsky.social @taccutexas.bsky.social @noirlabastro.bsky.social
Website astrovisbench.github.io
Paper arxiv.org/abs/2505.20538
@utaustin.bsky.social @simonsfoundation.org @uvapress.bsky.social @nyupress.bsky.social @taccutexas.bsky.social @noirlabastro.bsky.social
AstroVisbench · AstroVisBench
astrovisbench.github.io
September 25, 2025 at 7:07 PM
-are adequate at long-tail knowledge, focusing especially on the usage of domain-specific APIs and visualization generation
-interact with a variety of data formats to create diverse visualizations that comply with expert standards
-interact with a variety of data formats to create diverse visualizations that comply with expert standards
September 25, 2025 at 7:06 PM
-are adequate at long-tail knowledge, focusing especially on the usage of domain-specific APIs and visualization generation
-interact with a variety of data formats to create diverse visualizations that comply with expert standards
-interact with a variety of data formats to create diverse visualizations that comply with expert standards
AstroVisBench is the first scientific coding benchmark that evaluates whether models:
-aid scientists amidst their own workflows when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring.
-aid scientists amidst their own workflows when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring.
September 25, 2025 at 7:06 PM
AstroVisBench is the first scientific coding benchmark that evaluates whether models:
-aid scientists amidst their own workflows when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring.
-aid scientists amidst their own workflows when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring.
AstroVisBench was created by Sebastian Joseph, Syed Murtaza Husain, Stella Offner, @stephajuneau.bsky.social, Paul Torrey, Adam Bolton, Juan P. Farias, @niall2.bsky.social, @gregdnlp.bsky.social, and @jessyjli.bsky.social
September 25, 2025 at 7:04 PM
AstroVisBench was created by Sebastian Joseph, Syed Murtaza Husain, Stella Offner, @stephajuneau.bsky.social, Paul Torrey, Adam Bolton, Juan P. Farias, @niall2.bsky.social, @gregdnlp.bsky.social, and @jessyjli.bsky.social
✅ Accepted as part of the NeurIPS Datasets and Benchmarks Track 2025!
🔎 Findings: Even the best LLMs struggle to execute scientific workflows.
🔎 Findings: Even the best LLMs struggle to execute scientific workflows.
September 25, 2025 at 7:01 PM
✅ Accepted as part of the NeurIPS Datasets and Benchmarks Track 2025!
🔎 Findings: Even the best LLMs struggle to execute scientific workflows.
🔎 Findings: Even the best LLMs struggle to execute scientific workflows.