Katie Keith
@katakeith.bsky.social
NLP and computational social science (CSS) researcher. Assistant Professor in Computer Science at Williams College. AI2 and UMass Amherst alum. she/her. https://kakeith.github.io/
Reposted by Katie Keith
Very excited that my paper with @katakeith.bsky.social is now out in @polanalysis.bsky.social. We investigate whether LLMs actually follow the instructions/definitions provided in codebooks, propose some diagnostics, and release a new evaluation dataset.
www.cambridge.org/core/journal...
www.cambridge.org/core/journal...
Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts | Political Analysis | Cambridge Core
Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts
www.cambridge.org
September 19, 2025 at 1:45 PM
Very excited that my paper with @katakeith.bsky.social is now out in @polanalysis.bsky.social. We investigate whether LLMs actually follow the instructions/definitions provided in codebooks, propose some diagnostics, and release a new evaluation dataset.
www.cambridge.org/core/journal...
www.cambridge.org/core/journal...
Reposted by Katie Keith
The #ACL2025 #ACL2025NLP feed is up and running! It matches both hashtags and any posts from or mentions of @aclmeeting.bsky.social
Pin it to your home 📌 and enjoy!
bsky.app/profile/did:...
Pin it to your home 📌 and enjoy!
bsky.app/profile/did:...
July 17, 2025 at 11:15 AM
The #ACL2025 #ACL2025NLP feed is up and running! It matches both hashtags and any posts from or mentions of @aclmeeting.bsky.social
Pin it to your home 📌 and enjoy!
bsky.app/profile/did:...
Pin it to your home 📌 and enjoy!
bsky.app/profile/did:...
Topic @adeldaoud.bsky.social and I were discussing today at lunch at #ic2s2 and want to ask here:
What are the “known facts” in the social sciences? Which relationships between at least two social variables have been empirically found to have large effects and replicated by multiple groups?
What are the “known facts” in the social sciences? Which relationships between at least two social variables have been empirically found to have large effects and replicated by multiple groups?
July 24, 2025 at 12:57 PM
Topic @adeldaoud.bsky.social and I were discussing today at lunch at #ic2s2 and want to ask here:
What are the “known facts” in the social sciences? Which relationships between at least two social variables have been empirically found to have large effects and replicated by multiple groups?
What are the “known facts” in the social sciences? Which relationships between at least two social variables have been empirically found to have large effects and replicated by multiple groups?
Highlighting this thread. Based on what I'm seeing at #ic2s2 this week, this line of work is hot (if a bit crowded), but I predict will only be more widely adopted by social scientists in the future.
What are your favorite recent papers on using LMs for annotation (especially in a loop with human annotators), synthetic data for task-specific prediction, active learning, and similar?
Looking for practical methods for settings where human annotations are costly.
A few examples in thread ↴
Looking for practical methods for settings where human annotations are costly.
A few examples in thread ↴
July 23, 2025 at 1:07 PM
Highlighting this thread. Based on what I'm seeing at #ic2s2 this week, this line of work is hot (if a bit crowded), but I predict will only be more widely adopted by social scientists in the future.
This was top-down decision and Williams faculty have yet to formally discuss it. Unclear whether it is resistance or capitulation.
www.science.org/content/arti...
www.science.org/content/arti...
U.S. college is first to decline federal science grants because of new DEI language
Williams College says NSF and NIH requirement related to discrimination “undermines” academic freedom
www.science.org
June 6, 2025 at 9:48 PM
This was top-down decision and Williams faculty have yet to formally discuss it. Unclear whether it is resistance or capitulation.
www.science.org/content/arti...
www.science.org/content/arti...
Honored by the feature on my research, grant, and GPU cluster by the Williams magazine. today.williams.edu/magazine/a-c...
A Co-op for Computing
Faculty are diving into the exciting, data-crunching, AI world of GPMoo.
today.williams.edu
May 28, 2025 at 1:41 AM
Honored by the feature on my research, grant, and GPU cluster by the Williams magazine. today.williams.edu/magazine/a-c...
Reposted by Katie Keith
A full room for @katakeith.bsky.social's talk on proximal causal inference with text data ✨✨✨
January 27, 2025 at 11:19 PM
A full room for @katakeith.bsky.social's talk on proximal causal inference with text data ✨✨✨
Reposted by Katie Keith
Mark your calendars for these upcoming events tied to SCI and its One-U Responsible AI Initiative! Visit rai.utah.edu/events for details.
@parasharmanish.bsky.social @katakeith.bsky.social @anamarasovic.bsky.social @freiling.bsky.social
@parasharmanish.bsky.social @katakeith.bsky.social @anamarasovic.bsky.social @freiling.bsky.social
January 24, 2025 at 10:30 PM
Mark your calendars for these upcoming events tied to SCI and its One-U Responsible AI Initiative! Visit rai.utah.edu/events for details.
@parasharmanish.bsky.social @katakeith.bsky.social @anamarasovic.bsky.social @freiling.bsky.social
@parasharmanish.bsky.social @katakeith.bsky.social @anamarasovic.bsky.social @freiling.bsky.social
Check out our #NeurIPS2024 poster (presented by my collaborators Jacob Chen and Rohit Bhattacharya) about “Proximal Causal Inference With Text Data” at 5:30pm tomorrow (Weds)!
neurips.cc/virtual/2024...
neurips.cc/virtual/2024...
December 11, 2024 at 1:10 AM
Check out our #NeurIPS2024 poster (presented by my collaborators Jacob Chen and Rohit Bhattacharya) about “Proximal Causal Inference With Text Data” at 5:30pm tomorrow (Weds)!
neurips.cc/virtual/2024...
neurips.cc/virtual/2024...
Reposted by Katie Keith
We're hiring new #nlp faculty this year!
Asst or Assoc Professors in NLP at UMass CICS --
careers.umass.edu/amherst/en-u...
Asst or Assoc Professors in NLP at UMass CICS --
careers.umass.edu/amherst/en-u...
Details - Assistant/Associate Professor - Natural Language Processing (NLP) | Human Resources | UMass Amherst
careers.umass.edu
November 19, 2024 at 2:33 PM
We're hiring new #nlp faculty this year!
Asst or Assoc Professors in NLP at UMass CICS --
careers.umass.edu/amherst/en-u...
Asst or Assoc Professors in NLP at UMass CICS --
careers.umass.edu/amherst/en-u...
Reposted by Katie Keith
I'm excited to share that we've released v1.0 of our podcast corpus, SPoRC, led by my PhD student Ben Litterer! This first dataset is a slice of time, comprising over one million episodes from May and June 2020, including transcripts, diarization, and extracted audio features.
Podcasts are a popular medium, but data for computational research is limited! We introduce the Structured Podcast Research Corpus (SPoRC - huggingface.co/datasets/bli...), a large, multimodal dataset of English podcasts 🧵
arxiv.org/abs/2411.07892
arxiv.org/abs/2411.07892
Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus
Podcasts provide highly diverse content to a massive listener base through a unique on-demand modality. However, limited data has prevented large-scale computational analysis of the podcast ecosystem....
arxiv.org
November 15, 2024 at 3:02 PM
I'm excited to share that we've released v1.0 of our podcast corpus, SPoRC, led by my PhD student Ben Litterer! This first dataset is a slice of time, comprising over one million episodes from May and June 2020, including transcripts, diarization, and extracted audio features.
Reposted by Katie Keith
Starter packs are genius, but I was surprised there wasn't a list of them for people to find.
So I built it:
blueskydirectory.com/starter-pack...
The website monitors the packs being shared and adds the ones it finds to the database.
Missed your stater pack? Message me and I'll get it added.
So I built it:
blueskydirectory.com/starter-pack...
The website monitors the packs being shared and adds the ones it finds to the database.
Missed your stater pack? Message me and I'll get it added.
All - Bluesky Directory
A curated collection of all things relating to the Blue Sky social media platform.
blueskydirectory.com
November 11, 2024 at 4:13 PM
Starter packs are genius, but I was surprised there wasn't a list of them for people to find.
So I built it:
blueskydirectory.com/starter-pack...
The website monitors the packs being shared and adds the ones it finds to the database.
Missed your stater pack? Message me and I'll get it added.
So I built it:
blueskydirectory.com/starter-pack...
The website monitors the packs being shared and adds the ones it finds to the database.
Missed your stater pack? Message me and I'll get it added.
Reposted by Katie Keith
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
November 9, 2024 at 9:13 AM
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
In my NLP class (www.cs.williams.edu/~kkeith/teac...) next week, we're talking about eval.
I'd like to have a large section of the lecture focus on contamination. Crowd-sourcing--please send me your favorite contamination papers! Thanks! 🙏
I'd like to have a large section of the lecture focus on contamination. Crowd-sourcing--please send me your favorite contamination papers! Thanks! 🙏
www.cs.williams.edu
November 6, 2024 at 8:27 PM
In my NLP class (www.cs.williams.edu/~kkeith/teac...) next week, we're talking about eval.
I'd like to have a large section of the lecture focus on contamination. Crowd-sourcing--please send me your favorite contamination papers! Thanks! 🙏
I'd like to have a large section of the lecture focus on contamination. Crowd-sourcing--please send me your favorite contamination papers! Thanks! 🙏
Reposted by Katie Keith
November 5, 2024 at 9:39 PM
Reposted by Katie Keith
We are hosting the 11th International Conference on Computational Social Science in Sweden
🚀The IC2S2'25 website is LIVE, and submissions are OPEN!
📍Norrköping | July 21-24, 2025
Call for Abstracts (until Feb 24)
Call for Tutorials (until Jan 17)
🔗Explore details & submit: ic2s2-2025.org
🚀The IC2S2'25 website is LIVE, and submissions are OPEN!
📍Norrköping | July 21-24, 2025
Call for Abstracts (until Feb 24)
Call for Tutorials (until Jan 17)
🔗Explore details & submit: ic2s2-2025.org
October 25, 2024 at 12:10 PM
We are hosting the 11th International Conference on Computational Social Science in Sweden
🚀The IC2S2'25 website is LIVE, and submissions are OPEN!
📍Norrköping | July 21-24, 2025
Call for Abstracts (until Feb 24)
Call for Tutorials (until Jan 17)
🔗Explore details & submit: ic2s2-2025.org
🚀The IC2S2'25 website is LIVE, and submissions are OPEN!
📍Norrköping | July 21-24, 2025
Call for Abstracts (until Feb 24)
Call for Tutorials (until Jan 17)
🔗Explore details & submit: ic2s2-2025.org
Reposted by Katie Keith
November 4, 2024 at 10:01 AM
Our Williams College computer science department is hiring TWO TENURE-TRACK FACULTY this cycle: apply.interfolio.com/148956
(1/4)
(1/4)
August 20, 2024 at 7:39 PM
Our Williams College computer science department is hiring TWO TENURE-TRACK FACULTY this cycle: apply.interfolio.com/148956
(1/4)
(1/4)
Was introduced to Ted Chiang’s fiction recently and I can’t put it down! So thought provoking!
January 28, 2024 at 12:46 PM
Was introduced to Ted Chiang’s fiction recently and I can’t put it down! So thought provoking!
🚨NLP+CSS workshop is back and will be at NAACL 2024!
Paper submission deadline: March 24 sites.google.com/site/nlpandc...
Organizing team: Anjalie Field, @dallascard.bsky.social Dirk Hovy and myself
Paper submission deadline: March 24 sites.google.com/site/nlpandc...
Organizing team: Anjalie Field, @dallascard.bsky.social Dirk Hovy and myself
NLP+CSS Workshops
https://www.pexels.com/photo/group-hand-fist-bump-1068523/
sites.google.com
December 22, 2023 at 4:43 PM
🚨NLP+CSS workshop is back and will be at NAACL 2024!
Paper submission deadline: March 24 sites.google.com/site/nlpandc...
Organizing team: Anjalie Field, @dallascard.bsky.social Dirk Hovy and myself
Paper submission deadline: March 24 sites.google.com/site/nlpandc...
Organizing team: Anjalie Field, @dallascard.bsky.social Dirk Hovy and myself