Ken Holstein
@kenholstein.bsky.social
HCI Prof at CMU HCII. Research on augmented intelligence, participatory AI, & complementarity in human-human and human-AI workflows.
thecoalalab.com
thecoalalab.com
Reposted by Ken Holstein
I am beyond trilled to present two 🏆 award-winning papers 🏆 @acm-cscw.bsky.social this year CSCW in beautiful Bergen, Norway!! Broadly, I'm excited to chat about responsible AI, AI auditing and red-teaming, and human-AI interaction. Say hi if we run into each other!
October 19, 2025 at 1:49 PM
I am beyond trilled to present two 🏆 award-winning papers 🏆 @acm-cscw.bsky.social this year CSCW in beautiful Bergen, Norway!! Broadly, I'm excited to chat about responsible AI, AI auditing and red-teaming, and human-AI interaction. Say hi if we run into each other!
Reposted by Ken Holstein
Inspired by a post by @bayesianboy.bsky.social , I wrote something on the differences between information and insight — and why understanding remains a very human act
The point is to understand
Machines can’t make sense of the world for us.
renderghost.leaflet.pub
October 16, 2025 at 10:46 AM
Inspired by a post by @bayesianboy.bsky.social , I wrote something on the differences between information and insight — and why understanding remains a very human act
Reposted by Ken Holstein
A subtle aspect of predictive modeling is target variable construction: the process of translating a latent, unobservable concept like "healthcare need" into a prediction target
But how does target variable construction unfold in practice, and how can we better support it going forward? #CSCW2025 🧵
But how does target variable construction unfold in practice, and how can we better support it going forward? #CSCW2025 🧵
October 14, 2025 at 2:54 PM
A subtle aspect of predictive modeling is target variable construction: the process of translating a latent, unobservable concept like "healthcare need" into a prediction target
But how does target variable construction unfold in practice, and how can we better support it going forward? #CSCW2025 🧵
But how does target variable construction unfold in practice, and how can we better support it going forward? #CSCW2025 🧵
Reposted by Ken Holstein
LLM safety work often reasons over high-level policies (be helpful & polite), but must tackle on-the-ground cases (unsolicited money advice when stocks are mentioned). This can feel like driving on an unfamiliar road guided by a generic driver’s manual instead of a map. We introduce: Policy Maps 🗺️
September 29, 2025 at 3:54 PM
LLM safety work often reasons over high-level policies (be helpful & polite), but must tackle on-the-ground cases (unsolicited money advice when stocks are mentioned). This can feel like driving on an unfamiliar road guided by a generic driver’s manual instead of a map. We introduce: Policy Maps 🗺️
Reposted by Ken Holstein
There is a lot of talk and effort to figure out how genAI is different (I am also guilty of this!) -- the reality is that genAI is not that different and genAI is not that new either; it was hard to evaluate in the past, and it is still as hard to evaluate now #facct2025
June 23, 2025 at 7:17 AM
There is a lot of talk and effort to figure out how genAI is different (I am also guilty of this!) -- the reality is that genAI is not that different and genAI is not that new either; it was hard to evaluate in the past, and it is still as hard to evaluate now #facct2025
Reposted by Ken Holstein
Check out the camera-ready version of our ICML position paper ("Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge") to learn more!!! arxiv.org/abs/2502.00561
(6/6)
(6/6)
Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
The measurement tasks involved in evaluating generative AI (GenAI) systems lack sufficient scientific rigor, leading to what has been described as "a tangle of sloppy tests [and] apples-to-oranges com...
arxiv.org
June 15, 2025 at 12:20 AM
Check out the camera-ready version of our ICML position paper ("Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge") to learn more!!! arxiv.org/abs/2502.00561
(6/6)
(6/6)
Reposted by Ken Holstein
Somewhat oddly, the Trump regime's initial moves to cut science funding went very broad. This is catalyzing solidarity & advocacy around the value of science (see comms featuring cancer cures & tech innovation). But their next move may be to try to drive a wedge b/w "good science" & "bad science".
May 15, 2025 at 3:02 PM
Somewhat oddly, the Trump regime's initial moves to cut science funding went very broad. This is catalyzing solidarity & advocacy around the value of science (see comms featuring cancer cures & tech innovation). But their next move may be to try to drive a wedge b/w "good science" & "bad science".
Reposted by Ken Holstein
How's fascism going for you? medium.com/bits-and-beh...
How is fascism going for you? Here’s how it’s going for me.
A catalog of impacts thus far from this administration’s horrors.
medium.com
May 12, 2025 at 12:45 AM
How's fascism going for you? medium.com/bits-and-beh...
Reposted by Ken Holstein
Securing your phone before entering the USA. 🧵
At CDC we travel around the globe to confront health threats and have never worried about returning home. Yet more researchers, including US Citizens, are having their phones searched at Customs.
security.ucop.edu/resources/tr...
At CDC we travel around the globe to confront health threats and have never worried about returning home. Yet more researchers, including US Citizens, are having their phones searched at Customs.
security.ucop.edu/resources/tr...
Returning to the U.S.
security.ucop.edu
March 24, 2025 at 9:14 PM
Securing your phone before entering the USA. 🧵
At CDC we travel around the globe to confront health threats and have never worried about returning home. Yet more researchers, including US Citizens, are having their phones searched at Customs.
security.ucop.edu/resources/tr...
At CDC we travel around the globe to confront health threats and have never worried about returning home. Yet more researchers, including US Citizens, are having their phones searched at Customs.
security.ucop.edu/resources/tr...
Reposted by Ken Holstein
Unidentified men grabbing someone off the street and putting her in a car because she wrote an op-Ed. This as flatly authoritarian as anything we’ve seen in this country in a very long time.
Video of the international student at Tufts being arrested by "federal authorities" in Massachusetts has been released and it's terrifying.
They're not even uniformed officers. Just secret police thugs in hoodies and masks.
From WCVB: youtu.be/PuFIs7OkzYY
They're not even uniformed officers. Just secret police thugs in hoodies and masks.
From WCVB: youtu.be/PuFIs7OkzYY
March 26, 2025 at 5:41 PM
Unidentified men grabbing someone off the street and putting her in a car because she wrote an op-Ed. This as flatly authoritarian as anything we’ve seen in this country in a very long time.
Reposted by Ken Holstein
1. @alisongopnik.bsky.social, Cosma Shalizi, James Evans and myself have a new piece in Science on "AI" Large Models, pushing back against much of the collective wisdom about what they can and can't do. Official below, unpaywalled at henryfarrell.net/large-ai-mod... . So why this now?
Large AI models are cultural and social technologies
Implications draw on the history of transformative information systems from the past
www.science.org
March 14, 2025 at 12:57 PM
1. @alisongopnik.bsky.social, Cosma Shalizi, James Evans and myself have a new piece in Science on "AI" Large Models, pushing back against much of the collective wisdom about what they can and can't do. Official below, unpaywalled at henryfarrell.net/large-ai-mod... . So why this now?
Reposted by Ken Holstein
How can we help communities collaboratively shape policies that impact them?
In our #CHI2025 paper, we present PolicyCraft, a system that supports ✨collaborative policy design✨ through case-grounded deliberation.
(🧵/11)
In our #CHI2025 paper, we present PolicyCraft, a system that supports ✨collaborative policy design✨ through case-grounded deliberation.
(🧵/11)
March 10, 2025 at 1:50 PM
How can we help communities collaboratively shape policies that impact them?
In our #CHI2025 paper, we present PolicyCraft, a system that supports ✨collaborative policy design✨ through case-grounded deliberation.
(🧵/11)
In our #CHI2025 paper, we present PolicyCraft, a system that supports ✨collaborative policy design✨ through case-grounded deliberation.
(🧵/11)
Reposted by Ken Holstein
How can we better think and talk about human-like qualities attributed to language technologies like LLMs? In our #CHI2025 paper, we taxonomize how text outputs from cases of user interactions with language technologies can contribute to anthropomorphism. arxiv.org/abs/2502.09870 1/n
March 6, 2025 at 3:43 AM
How can we better think and talk about human-like qualities attributed to language technologies like LLMs? In our #CHI2025 paper, we taxonomize how text outputs from cases of user interactions with language technologies can contribute to anthropomorphism. arxiv.org/abs/2502.09870 1/n
Reposted by Ken Holstein
"18F works just how Musk + team pretend that they want gov't to work. But when his team found it, they destroyed it. 18F’s work is evidence that gov't works well, which undermines their msg that it doesn’t. 18F’s parent agency, the [GSA], turns a profit as an agency. So it has to be destroyed too."
Requiem for 18F
A small government unit did what DOGE is pretending to do
donmoynihan.substack.com
March 4, 2025 at 3:34 PM
"18F works just how Musk + team pretend that they want gov't to work. But when his team found it, they destroyed it. 18F’s work is evidence that gov't works well, which undermines their msg that it doesn’t. 18F’s parent agency, the [GSA], turns a profit as an agency. So it has to be destroyed too."
Reposted by Ken Holstein
#Pittsburgh friends, Join me on Friday #standupforscience2025. Our city will be devastated if our universities lose critical funding. Our universities are top 10 employers, the entire city’s economy is dependent on science funding. www.eventbrite.com/e/stand-up-f...
Stand Up for Science 2025 - Pittsburgh, PA
Stand up for science with us on March 7th, 2025, because science is for everyone! More info at www.standupforscience2025.org
www.eventbrite.com
March 3, 2025 at 1:39 AM
#Pittsburgh friends, Join me on Friday #standupforscience2025. Our city will be devastated if our universities lose critical funding. Our universities are top 10 employers, the entire city’s economy is dependent on science funding. www.eventbrite.com/e/stand-up-f...
Reposted by Ken Holstein
🏛️ How can we ensure that AI enhances, rather than harms, public services?
In our policy memo, published with the Federation of American Scientists (@scientistsorg.bsky.social), we provide recommendations for how the fed gov't could help state & local agencies make better AI procurement decisions.🧵
In our policy memo, published with the Federation of American Scientists (@scientistsorg.bsky.social), we provide recommendations for how the fed gov't could help state & local agencies make better AI procurement decisions.🧵
Expanding Local Government Capacity for AI Procurement and Use
As new waves of AI technologies continue to enter the public sector, touching a breadth of services critical to the welfare of the American people, this center of excellence will help maintain high st...
fas.org
February 26, 2025 at 7:58 PM
🏛️ How can we ensure that AI enhances, rather than harms, public services?
In our policy memo, published with the Federation of American Scientists (@scientistsorg.bsky.social), we provide recommendations for how the fed gov't could help state & local agencies make better AI procurement decisions.🧵
In our policy memo, published with the Federation of American Scientists (@scientistsorg.bsky.social), we provide recommendations for how the fed gov't could help state & local agencies make better AI procurement decisions.🧵
Reposted by Ken Holstein
Joseph Ura: “A president refusing to abide by the law or the Constitution and ignoring court orders to stop his illegitimate actions would be a constitutional crisis like a bank robbery is a cash flow crisis.” www.theatlantic.com/newsletters/...
‘Constitutional Crisis’ Is an Understatement
It’s the hottest, and most useless, buzz phrase of the moment.
www.theatlantic.com
February 23, 2025 at 12:01 AM
Joseph Ura: “A president refusing to abide by the law or the Constitution and ignoring court orders to stop his illegitimate actions would be a constitutional crisis like a bank robbery is a cash flow crisis.” www.theatlantic.com/newsletters/...
Reposted by Ken Holstein
Friendly reminder to get your free Covid tests before they throw them away.
Covidtests.gov
wapo.st/42VLGQh
Covidtests.gov
wapo.st/42VLGQh
U.S. weighs destroying $500 million in stockpiled covid tests
The government is reviewing proposals to shut down the program that ships free covid tests to American households and has been considering destroying 160 million tests.
wapo.st
February 18, 2025 at 9:36 PM
Friendly reminder to get your free Covid tests before they throw them away.
Covidtests.gov
wapo.st/42VLGQh
Covidtests.gov
wapo.st/42VLGQh
Reposted by Ken Holstein
for pity’s sake, I am BEGGING university leaders to stop framing this issue in terms of what this will cost the university. people dont care!
EXPLAIN WHAT THIS WILL COST THE PUBLIC: closed hospitals and clinics, skipped treatments, loss of access to experimental drugs, unemployment, recession
EXPLAIN WHAT THIS WILL COST THE PUBLIC: closed hospitals and clinics, skipped treatments, loss of access to experimental drugs, unemployment, recession
February 9, 2025 at 10:00 PM
for pity’s sake, I am BEGGING university leaders to stop framing this issue in terms of what this will cost the university. people dont care!
EXPLAIN WHAT THIS WILL COST THE PUBLIC: closed hospitals and clinics, skipped treatments, loss of access to experimental drugs, unemployment, recession
EXPLAIN WHAT THIS WILL COST THE PUBLIC: closed hospitals and clinics, skipped treatments, loss of access to experimental drugs, unemployment, recession
Reposted by Ken Holstein
"AI ... becomes a tool for replacing politics. The Trump administration frames generative #AI as a remedy to 'government waste.' However, what it seeks to automate is not paperwork but democratic decision-making."
My latest for @techpolicypress.bsky.social, “Anatomy of an AI Coup.” With the takeover of the US government by tech elites underway, we must examine its goals and next steps — and how we will know if it has succeeded. www.techpolicy.press/anatomy-of-a...
Anatomy of an AI Coup | TechPolicy.Press
DOGE is gutting federal agencies to install AI across the government. Democracy is on the line, writes Tech Policy Press fellow Eryk Salvaggio.
www.techpolicy.press
February 9, 2025 at 7:02 AM
"AI ... becomes a tool for replacing politics. The Trump administration frames generative #AI as a remedy to 'government waste.' However, what it seeks to automate is not paperwork but democratic decision-making."
Reposted by Ken Holstein
Scientists and journalists need to figure out right quick how to explain to the average person how a massive change in research indirects will impact the medical care they and their children get (eg at the local children’s hospital), the education their children will get, the price of tuition, etc.
February 8, 2025 at 1:35 AM
Scientists and journalists need to figure out right quick how to explain to the average person how a massive change in research indirects will impact the medical care they and their children get (eg at the local children’s hospital), the education their children will get, the price of tuition, etc.
Reposted by Ken Holstein
What I should have done in the beginning - here's a starter pack of 20-odd experts who journalists or others might want to read/talk to to understand the gutting of the administrative state (as opposed to broad decline of democracy etc). I've focused on academics go.bsky.app/MaC7vnv
February 1, 2025 at 2:30 PM
What I should have done in the beginning - here's a starter pack of 20-odd experts who journalists or others might want to read/talk to to understand the gutting of the administrative state (as opposed to broad decline of democracy etc). I've focused on academics go.bsky.app/MaC7vnv
Reposted by Ken Holstein
First off, it's critical to note:
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
January 28, 2025 at 2:09 AM
First off, it's critical to note:
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
NSF is CONGRESSIONALLY MANDATED to evaluate proposals via broader impacts, as defined by this law below.
Hence, if the admin wants NSF to deviate, that's called **breaking the law** (important to remind Congress of that)
www.law.cornell.edu/uscode/text/...
2/5
Reposted by Ken Holstein
I'm an adventurous eater, but even I draw the line here
May 24, 2023 at 4:51 PM
I'm an adventurous eater, but even I draw the line here
Reposted by Ken Holstein
The moderation approach with custom labelers etc in Bluesky is pretty clever and worth understanding when you use this platform: bsky.social/about/blog/0....
Bluesky’s Stackable Approach to Moderation - Bluesky
Today, we’re open sourcing Ozone, a tool that lets a team of moderators or curators collaboratively review reports, create labels, and inspect content on the atproto network. Later this week, we’re op...
bsky.social
January 25, 2025 at 3:16 PM
The moderation approach with custom labelers etc in Bluesky is pretty clever and worth understanding when you use this platform: bsky.social/about/blog/0....