Ken Holstein
@kenholstein.bsky.social
HCI Prof at CMU HCII. Research on augmented intelligence, participatory AI, & complementarity in human-human and human-AI workflows.
thecoalalab.com
thecoalalab.com
Reposted by Ken Holstein
Check out the camera-ready version of our ICML position paper ("Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge") to learn more!!! arxiv.org/abs/2502.00561
(6/6)
(6/6)
Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
The measurement tasks involved in evaluating generative AI (GenAI) systems lack sufficient scientific rigor, leading to what has been described as "a tangle of sloppy tests [and] apples-to-oranges com...
arxiv.org
June 15, 2025 at 12:20 AM
Check out the camera-ready version of our ICML position paper ("Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge") to learn more!!! arxiv.org/abs/2502.00561
(6/6)
(6/6)
Reposted by Ken Holstein
First off, it's critical to note:
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
January 28, 2025 at 2:09 AM
First off, it's critical to note:
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5