Hal Daumé III
@haldaume3.bsky.social
Human-centered AI #HCAI, NLP & ML. Director TRAILS (Trustworthy AI in Law & Society) and AIM (AI Interdisciplinary Institute at Maryland). Formerly Microsoft Research NYC. Fun: 🧗🧑🍳🧘⛷️🏕️. he/him.
my teenagehood destroyed: pbrush now has AI support.
June 23, 2025 at 9:36 AM
my teenagehood destroyed: pbrush now has AI support.
There is a new version of the Research Plan for NIST's AI Safety Consortium (AISIC) in response to EOs. I did a diff.
Out: safety, responsibility, sociotechnical, fairness, working w fed agencies, authenticating content, watermarking, RN of CBRN, autonomous replication, ctrl of physical systems
>
Out: safety, responsibility, sociotechnical, fairness, working w fed agencies, authenticating content, watermarking, RN of CBRN, autonomous replication, ctrl of physical systems
>
March 4, 2025 at 12:29 PM
There is a new version of the Research Plan for NIST's AI Safety Consortium (AISIC) in response to EOs. I did a diff.
Out: safety, responsibility, sociotechnical, fairness, working w fed agencies, authenticating content, watermarking, RN of CBRN, autonomous replication, ctrl of physical systems
>
Out: safety, responsibility, sociotechnical, fairness, working w fed agencies, authenticating content, watermarking, RN of CBRN, autonomous replication, ctrl of physical systems
>
at TRAILCon today, enjoying TRAILS branded Trail Mix
February 3, 2025 at 3:03 PM
at TRAILCon today, enjoying TRAILS branded Trail Mix
Please join us for:
AI at Work: Building and Evaluating Trust
Presented by our Trustworthy AI in Law & Society (TRIALS) institute.
Feb 3-4
Washington DC
Open to all!
Details and registration at: trails.gwu.edu/trailscon-2025
Sponsorship details at: trails.gwu.edu/media/556
AI at Work: Building and Evaluating Trust
Presented by our Trustworthy AI in Law & Society (TRIALS) institute.
Feb 3-4
Washington DC
Open to all!
Details and registration at: trails.gwu.edu/trailscon-2025
Sponsorship details at: trails.gwu.edu/media/556
January 16, 2025 at 3:22 PM
Please join us for:
AI at Work: Building and Evaluating Trust
Presented by our Trustworthy AI in Law & Society (TRIALS) institute.
Feb 3-4
Washington DC
Open to all!
Details and registration at: trails.gwu.edu/trailscon-2025
Sponsorship details at: trails.gwu.edu/media/556
AI at Work: Building and Evaluating Trust
Presented by our Trustworthy AI in Law & Society (TRIALS) institute.
Feb 3-4
Washington DC
Open to all!
Details and registration at: trails.gwu.edu/trailscon-2025
Sponsorship details at: trails.gwu.edu/media/556
the previous is "what is intelligence" which then transitioned to "what is AI":
here, i kind of like kaplan+haenlein and also google. tesler is of course always correct and may really be the right answer.
here, i kind of like kaplan+haenlein and also google. tesler is of course always correct and may really be the right answer.
![Poll results:
McCarthy (1956) "The science and engineering of making intelligent machines: [it is] the computational part of the ability to achieve goals in the world." - 3 votes
Marvin Minsky (1956) "The construction of computer programs that engage in tasks that are currently more satisfactorily performed by human beings because they require...: perceptual learning, memory organization and critical reasoning." - 19 votes
John Searle (1960s) "The appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds." - 17 votes
Kaplan & Haenlein (2019) "A system's ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation." - 60 votes
Google (2020s) "AI is a field of science concerned with building computers and machines that can reason, learn, and act in such a way that would normally require human intelligence or that involves data whose scale exceeds what humans can analyze." - 71 votes
Larry Tesler (1980s, debated) "AI is whatever hasn't been done yet." - 2 votes](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:aexevhwfk5oz5vmv4bljwows/bafkreidv7gy74z2kryzc4brf7jsnaaivnvvbafzkkl43xcuy5hm24gd3z4@jpeg)
December 18, 2024 at 3:22 PM
the previous is "what is intelligence" which then transitioned to "what is AI":
here, i kind of like kaplan+haenlein and also google. tesler is of course always correct and may really be the right answer.
here, i kind of like kaplan+haenlein and also google. tesler is of course always correct and may really be the right answer.
fwiw, I did a poll with undergrads (this was in the second day of a gened genai course) with seven definitions (all with citations inline).
i kind of like gardner and legg+hutter fwiw.
i kind of like gardner and legg+hutter fwiw.
![Poll results.
Alfred Binet: Judgment, otherwise called "good sense", "practical sense", "initiative", the faculty of adapting one's self to circumstances ... auto-critique. - 15 votes
David Wechsler: The aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment. - 26 votes
Lloyd Humphreys: The resultant of the process of acquiring, storing in memory, retrieving, combining, comparing, and using in new contexts information and conceptual skills. - 49 votes
Howard Gardner: [Intelligence] entail[s] a set of skills of problem solving—enabling the individual to resolve genuine problems ... they encounter[] —... and the potential for finding or creating problems. - 41 votes
Robert Sternberg & William Salter: Goal-directed adaptive behavior. - 8 votes
Reuven Feuerstein: The unique propensity of human beings to change or modify the structure of their cognitive functioning to adapt to the changing demands of a life situation. - 23 votes
Shane Legg & Marcus Hutter: Intelligence measures an agent's ability to achieve goals in a wide range of environments. - 10 votes](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:aexevhwfk5oz5vmv4bljwows/bafkreifwo34zcsfgriywotiemyy5fqh2a7bzk6lphisrsclv7aazqukaa4@jpeg)
December 18, 2024 at 3:18 PM
fwiw, I did a poll with undergrads (this was in the second day of a gened genai course) with seven definitions (all with citations inline).
i kind of like gardner and legg+hutter fwiw.
i kind of like gardner and legg+hutter fwiw.
for the linguists out there
![modified distracted boyfriend meme. he's looking away from two women, one labeled [pa], one saying [ka], and looking toward another labeled [ta].](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:aexevhwfk5oz5vmv4bljwows/bafkreigz775ovalop7bb6cfber2ucb6xo6dksvpng4iittybffm35sezgm@jpeg)
December 18, 2024 at 8:19 AM
for the linguists out there
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
We show that a carefully constructed temporal contrastive loss leads to effective, multitask RL pretraining.
by Ruijie Zheng +al ICML’24
hal3.name/docs/daume24...
(eos)
We show that a carefully constructed temporal contrastive loss leads to effective, multitask RL pretraining.
by Ruijie Zheng +al ICML’24
hal3.name/docs/daume24...
(eos)
December 16, 2024 at 10:51 AM
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
We show that a carefully constructed temporal contrastive loss leads to effective, multitask RL pretraining.
by Ruijie Zheng +al ICML’24
hal3.name/docs/daume24...
(eos)
We show that a carefully constructed temporal contrastive loss leads to effective, multitask RL pretraining.
by Ruijie Zheng +al ICML’24
hal3.name/docs/daume24...
(eos)
HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via LLMs
There are lots of hate datasets with different nuances. We show how to pretrain on a COT-enhanced dataset to get great performance on data du jour.
by Huy Nghiem +al EMNLP’24
hal3.name/docs/daume24...
>
There are lots of hate datasets with different nuances. We show how to pretrain on a COT-enhanced dataset to get great performance on data du jour.
by Huy Nghiem +al EMNLP’24
hal3.name/docs/daume24...
>
December 13, 2024 at 10:20 AM
HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via LLMs
There are lots of hate datasets with different nuances. We show how to pretrain on a COT-enhanced dataset to get great performance on data du jour.
by Huy Nghiem +al EMNLP’24
hal3.name/docs/daume24...
>
There are lots of hate datasets with different nuances. We show how to pretrain on a COT-enhanced dataset to get great performance on data du jour.
by Huy Nghiem +al EMNLP’24
hal3.name/docs/daume24...
>
starting to feel like slack is as bad as email.
December 12, 2024 at 1:22 PM
starting to feel like slack is as bad as email.
Do great minds think alike? Investigating Human-AI Complementarity in QA
We use item response theory to compare the capabilities of 155 people vs 70 chatbots at answering questions, teasing apart complementarities; implications for design.
by Maharshi Gor +al EMNLP’24
hal3.name/docs/daume24...
>
We use item response theory to compare the capabilities of 155 people vs 70 chatbots at answering questions, teasing apart complementarities; implications for design.
by Maharshi Gor +al EMNLP’24
hal3.name/docs/daume24...
>
December 12, 2024 at 10:41 AM
Do great minds think alike? Investigating Human-AI Complementarity in QA
We use item response theory to compare the capabilities of 155 people vs 70 chatbots at answering questions, teasing apart complementarities; implications for design.
by Maharshi Gor +al EMNLP’24
hal3.name/docs/daume24...
>
We use item response theory to compare the capabilities of 155 people vs 70 chatbots at answering questions, teasing apart complementarities; implications for design.
by Maharshi Gor +al EMNLP’24
hal3.name/docs/daume24...
>
Understanding the Impacts of Language Technologies’ Performance Disparities on AAL Speakers
We find AAL speakers expend significant invisible labor in order to achieve parity of outputs in LT systems; fairness measures don't capture this.
by Jay Cunningham +al ACL’24
hal3.name/docs/daume24...
>
We find AAL speakers expend significant invisible labor in order to achieve parity of outputs in LT systems; fairness measures don't capture this.
by Jay Cunningham +al ACL’24
hal3.name/docs/daume24...
>
December 11, 2024 at 8:53 AM
Understanding the Impacts of Language Technologies’ Performance Disparities on AAL Speakers
We find AAL speakers expend significant invisible labor in order to achieve parity of outputs in LT systems; fairness measures don't capture this.
by Jay Cunningham +al ACL’24
hal3.name/docs/daume24...
>
We find AAL speakers expend significant invisible labor in order to achieve parity of outputs in LT systems; fairness measures don't capture this.
by Jay Cunningham +al ACL’24
hal3.name/docs/daume24...
>
DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization
We show in RL that explicitly minimizing the dormant ratio (fraction of non-active neurons) improves exploration, rewards, etc.
by Guowei Xu, Ruijie Zheng, Yongyuan Liang +al ICLR’24
hal3.name/docs/daume24...
>
We show in RL that explicitly minimizing the dormant ratio (fraction of non-active neurons) improves exploration, rewards, etc.
by Guowei Xu, Ruijie Zheng, Yongyuan Liang +al ICLR’24
hal3.name/docs/daume24...
>
December 10, 2024 at 9:21 AM
DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization
We show in RL that explicitly minimizing the dormant ratio (fraction of non-active neurons) improves exploration, rewards, etc.
by Guowei Xu, Ruijie Zheng, Yongyuan Liang +al ICLR’24
hal3.name/docs/daume24...
>
We show in RL that explicitly minimizing the dormant ratio (fraction of non-active neurons) improves exploration, rewards, etc.
by Guowei Xu, Ruijie Zheng, Yongyuan Liang +al ICLR’24
hal3.name/docs/daume24...
>
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Despite hopes that explanations improve fairness, we see that when biases are hidden behind proxy features, explanations may not help.
Navita Goyal, Connor Baumler +al IUI’24
hal3.name/docs/daume23...
>
Despite hopes that explanations improve fairness, we see that when biases are hidden behind proxy features, explanations may not help.
Navita Goyal, Connor Baumler +al IUI’24
hal3.name/docs/daume23...
>
December 9, 2024 at 11:41 AM
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Despite hopes that explanations improve fairness, we see that when biases are hidden behind proxy features, explanations may not help.
Navita Goyal, Connor Baumler +al IUI’24
hal3.name/docs/daume23...
>
Despite hopes that explanations improve fairness, we see that when biases are hidden behind proxy features, explanations may not help.
Navita Goyal, Connor Baumler +al IUI’24
hal3.name/docs/daume23...
>
PRISE: Learning Temporal Action Abstractions as a Sequence Compression Problem
We show that LLM-style byte-pair encoding can be used to compress action sequences to give “meta-actions” of different lengths, improving RL performance.
by Ruijie Zheng +al ICML ‘24
hal3.name/docs/daume24...
>
We show that LLM-style byte-pair encoding can be used to compress action sequences to give “meta-actions” of different lengths, improving RL performance.
by Ruijie Zheng +al ICML ‘24
hal3.name/docs/daume24...
>
December 6, 2024 at 9:06 AM
PRISE: Learning Temporal Action Abstractions as a Sequence Compression Problem
We show that LLM-style byte-pair encoding can be used to compress action sequences to give “meta-actions” of different lengths, improving RL performance.
by Ruijie Zheng +al ICML ‘24
hal3.name/docs/daume24...
>
We show that LLM-style byte-pair encoding can be used to compress action sequences to give “meta-actions” of different lengths, improving RL performance.
by Ruijie Zheng +al ICML ‘24
hal3.name/docs/daume24...
>
An RCT on Anonymizing Reviewers to Each Other in Peer Review Discussions
We observe that anon discussions between revs led to slightly more discussion, less influence of seniority, no diff in politeness, & is slightly preferred by revs.
by Charvi Rastogi et al PLOS1’24
hal3.name/docs/daume24...
>
We observe that anon discussions between revs led to slightly more discussion, less influence of seniority, no diff in politeness, & is slightly preferred by revs.
by Charvi Rastogi et al PLOS1’24
hal3.name/docs/daume24...
>
December 5, 2024 at 9:03 AM
An RCT on Anonymizing Reviewers to Each Other in Peer Review Discussions
We observe that anon discussions between revs led to slightly more discussion, less influence of seniority, no diff in politeness, & is slightly preferred by revs.
by Charvi Rastogi et al PLOS1’24
hal3.name/docs/daume24...
>
We observe that anon discussions between revs led to slightly more discussion, less influence of seniority, no diff in politeness, & is slightly preferred by revs.
by Charvi Rastogi et al PLOS1’24
hal3.name/docs/daume24...
>
"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of LLMs in Employment Recommendations
Chatbots routinely prefer candidates with White, female-sounding names over others, even when candidates have identical qualifications.
by Huy Nghiem et al EMNLP’24
hal3.name/docs/daume24...
>
Chatbots routinely prefer candidates with White, female-sounding names over others, even when candidates have identical qualifications.
by Huy Nghiem et al EMNLP’24
hal3.name/docs/daume24...
>
December 4, 2024 at 9:02 AM
"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of LLMs in Employment Recommendations
Chatbots routinely prefer candidates with White, female-sounding names over others, even when candidates have identical qualifications.
by Huy Nghiem et al EMNLP’24
hal3.name/docs/daume24...
>
Chatbots routinely prefer candidates with White, female-sounding names over others, even when candidates have identical qualifications.
by Huy Nghiem et al EMNLP’24
hal3.name/docs/daume24...
>
Large Language Models Help Humans Verify Truthfulness—Except When They Are Convincingly Wrong
Should one use chatbots or web search to fact check? Chatbots help more on avg, but people uncritically accept their suggestions much more often.
by Chenglei Si +al NAACL’24
hal3.name/docs/daume24...
>
Should one use chatbots or web search to fact check? Chatbots help more on avg, but people uncritically accept their suggestions much more often.
by Chenglei Si +al NAACL’24
hal3.name/docs/daume24...
>
December 3, 2024 at 9:31 AM
Large Language Models Help Humans Verify Truthfulness—Except When They Are Convincingly Wrong
Should one use chatbots or web search to fact check? Chatbots help more on avg, but people uncritically accept their suggestions much more often.
by Chenglei Si +al NAACL’24
hal3.name/docs/daume24...
>
Should one use chatbots or web search to fact check? Chatbots help more on avg, but people uncritically accept their suggestions much more often.
by Chenglei Si +al NAACL’24
hal3.name/docs/daume24...
>
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections
When generating instructions for people, we can help them by highlighting potential confabs, AND by suggesting alternatives.
by Lingjun Zhao EMNLP’24
hal3.name/docs/daume24...
>
When generating instructions for people, we can help them by highlighting potential confabs, AND by suggesting alternatives.
by Lingjun Zhao EMNLP’24
hal3.name/docs/daume24...
>
December 2, 2024 at 9:37 AM
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections
When generating instructions for people, we can help them by highlighting potential confabs, AND by suggesting alternatives.
by Lingjun Zhao EMNLP’24
hal3.name/docs/daume24...
>
When generating instructions for people, we can help them by highlighting potential confabs, AND by suggesting alternatives.
by Lingjun Zhao EMNLP’24
hal3.name/docs/daume24...
>
france: no we don’t do halloween because it’s too american and commercialized
also france: you’ve heard of black friday? let me introduce you to
also france: you’ve heard of black friday? let me introduce you to
December 1, 2024 at 7:52 AM
france: no we don’t do halloween because it’s too american and commercialized
also france: you’ve heard of black friday? let me introduce you to
also france: you’ve heard of black friday? let me introduce you to
ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles
We develop a continuous signing dataset for ASL on a STEM subset of Wikipedia; challenges suggest problems related to fingerspelling detection, sign linking, & translation.
by Kayo Yin et al EMNLP’24
hal3.name/docs/daume24...
>
We develop a continuous signing dataset for ASL on a STEM subset of Wikipedia; challenges suggest problems related to fingerspelling detection, sign linking, & translation.
by Kayo Yin et al EMNLP’24
hal3.name/docs/daume24...
>
November 27, 2024 at 9:00 AM
ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles
We develop a continuous signing dataset for ASL on a STEM subset of Wikipedia; challenges suggest problems related to fingerspelling detection, sign linking, & translation.
by Kayo Yin et al EMNLP’24
hal3.name/docs/daume24...
>
We develop a continuous signing dataset for ASL on a STEM subset of Wikipedia; challenges suggest problems related to fingerspelling detection, sign linking, & translation.
by Kayo Yin et al EMNLP’24
hal3.name/docs/daume24...
>
Join fellow members of the AI Safety Institute Consortium (AISIC) on Dec 3 for the 1st annual plenary, hosted by TRAILS on behalf of NIST at UMD. (Reception on Dec 2 evening.)
This event is only open to individuals who work for a member of AISIC.
📅Register by Dec 2
👉Learn more: go.umd.edu/1u0n
This event is only open to individuals who work for a member of AISIC.
📅Register by Dec 2
👉Learn more: go.umd.edu/1u0n
November 27, 2024 at 8:58 AM
Join fellow members of the AI Safety Institute Consortium (AISIC) on Dec 3 for the 1st annual plenary, hosted by TRAILS on behalf of NIST at UMD. (Reception on Dec 2 evening.)
This event is only open to individuals who work for a member of AISIC.
📅Register by Dec 2
👉Learn more: go.umd.edu/1u0n
This event is only open to individuals who work for a member of AISIC.
📅Register by Dec 2
👉Learn more: go.umd.edu/1u0n
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators
Much work aims to help moderators by automation; much focusing on toxicity. Turns out that’s a small part of what volunteer mods want.
by Yang Trista Cao +al EMNLP’24
hal3.name/docs/daume24...
>
Much work aims to help moderators by automation; much focusing on toxicity. Turns out that’s a small part of what volunteer mods want.
by Yang Trista Cao +al EMNLP’24
hal3.name/docs/daume24...
>
November 26, 2024 at 8:45 AM
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators
Much work aims to help moderators by automation; much focusing on toxicity. Turns out that’s a small part of what volunteer mods want.
by Yang Trista Cao +al EMNLP’24
hal3.name/docs/daume24...
>
Much work aims to help moderators by automation; much focusing on toxicity. Turns out that’s a small part of what volunteer mods want.
by Yang Trista Cao +al EMNLP’24
hal3.name/docs/daume24...
>
How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?
Authors overestimate the chances their papers'll be accepted; co-auths disagree about paper value about as much as auth/revs.
by Charvi Rastogi +al PLOS One'24
hal3.name/docs/daume24...
>
Authors overestimate the chances their papers'll be accepted; co-auths disagree about paper value about as much as auth/revs.
by Charvi Rastogi +al PLOS One'24
hal3.name/docs/daume24...
>
November 25, 2024 at 9:34 AM
How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?
Authors overestimate the chances their papers'll be accepted; co-auths disagree about paper value about as much as auth/revs.
by Charvi Rastogi +al PLOS One'24
hal3.name/docs/daume24...
>
Authors overestimate the chances their papers'll be accepted; co-auths disagree about paper value about as much as auth/revs.
by Charvi Rastogi +al PLOS One'24
hal3.name/docs/daume24...
>