@boydgraber.bsky.social
I think you mean: Terror has spread. Even in The Loop, hundreds run from gunshots. Traffic has ground to a halt. Unable to do anything to prevent the stampede, hundreds scream from the sidewalks. "Police" stand by, doing nothing.
October 13, 2025 at 2:30 PM
I think you mean: Terror has spread. Even in The Loop, hundreds run from gunshots. Traffic has ground to a halt. Unable to do anything to prevent the stampede, hundreds scream from the sidewalks. "Police" stand by, doing nothing.
We had come at it more from the position of trying to use as few dev examples as possible (to keep them secret). I.e., use the best items you could and every model uses the exact same. But it makes sense to use the adaptive testing scenario if you don't mind potentially exposing more dev.
September 18, 2025 at 8:23 PM
We had come at it more from the position of trying to use as few dev examples as possible (to keep them secret). I.e., use the best items you could and every model uses the exact same. But it makes sense to use the adaptive testing scenario if you don't mind potentially exposing more dev.
In 2021, we proposed using IRT to find bad examples and to create more targeted leaderboards (Evaluation
Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?).
From my reading, the big difference seems to be that they're also using the agent's skill, which is super cool!
Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?).
From my reading, the big difference seems to be that they're also using the agent's skill, which is super cool!
September 18, 2025 at 8:19 PM
In 2021, we proposed using IRT to find bad examples and to create more targeted leaderboards (Evaluation
Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?).
From my reading, the big difference seems to be that they're also using the agent's skill, which is super cool!
Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?).
From my reading, the big difference seems to be that they're also using the agent's skill, which is super cool!
We also found that it's helpful for improving uncertainty estimation of models:
arxiv.org/abs/2205.12507
arxiv.org/abs/2205.12507
arxiv.org
September 18, 2025 at 8:13 PM
We also found that it's helpful for improving uncertainty estimation of models:
arxiv.org/abs/2205.12507
arxiv.org/abs/2205.12507
If it said that 1990 was "about 10 years ago", I would say that it has reached tenured faculty-level intelligence.
September 2, 2025 at 4:09 PM
If it said that 1990 was "about 10 years ago", I would say that it has reached tenured faculty-level intelligence.
A couple of weeks ago I left my family behind at a cable car station to finish climbing to the peak of a mountain because they were too scared to continue. When I reached the top, my phone gave a notification: new podcasts available for download. Apparently LMU has an observatory on Wendelstein.
August 21, 2025 at 2:45 PM
A couple of weeks ago I left my family behind at a cable car station to finish climbing to the peak of a mountain because they were too scared to continue. When I reached the top, my phone gave a notification: new podcasts available for download. Apparently LMU has an observatory on Wendelstein.
Do you mean salary, physical facilities, work environment, or funding ecosystem?
August 4, 2025 at 2:17 PM
Do you mean salary, physical facilities, work environment, or funding ecosystem?
At the risk of picking out one of my favorite children, this was the paper with our best traditional video of this cycle (thanks to Jon May for playing along):
t.co/QQlgwzo6jf
t.co/2G6kwAAPMy
t.co/QQlgwzo6jf
t.co/2G6kwAAPMy
https://youtu.be/L_hcHQep3fc
t.co
July 28, 2025 at 8:35 AM
At the risk of picking out one of my favorite children, this was the paper with our best traditional video of this cycle (thanks to Jon May for playing along):
t.co/QQlgwzo6jf
t.co/2G6kwAAPMy
t.co/QQlgwzo6jf
t.co/2G6kwAAPMy
Finally, this evening I'll be standing in for
@wwongkamjan.bsky.social
at the Findings poster (18:00, Hall X4/X5): Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
x.com/joywwong/sta...
@wwongkamjan.bsky.social
at the Findings poster (18:00, Hall X4/X5): Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
x.com/joywwong/sta...
Joy Wongkamjan on X: "Our paper CTRL-D is accepted to ACL Findings and will be presented at ACL 2025! 🗓️Poster session: 18:00–19:30 (Level 0 Exhibit Halls X4/X5) I’m sad I can’t be there, but Jordan (@boydgraber) will! You’ll enjoy learning about CTRL-D from him. Now… what is CTRL-D? 🔍 https://t.co/ucIPZRHBF1" / X
Our paper CTRL-D is accepted to ACL Findings and will be presented at ACL 2025! 🗓️Poster session: 18:00–19:30 (Level 0 Exhibit Halls X4/X5) I’m sad I can’t be there, but Jordan (@boydgraber) will! You’ll enjoy learning about CTRL-D from him. Now… what is CTRL-D? 🔍 https://t.co/ucIPZRHBF1
x.com
July 28, 2025 at 8:35 AM
Finally, this evening I'll be standing in for
@wwongkamjan.bsky.social
at the Findings poster (18:00, Hall X4/X5): Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
x.com/joywwong/sta...
@wwongkamjan.bsky.social
at the Findings poster (18:00, Hall X4/X5): Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
x.com/joywwong/sta...
In the second oral paper (14:22 PM, Room 1.62),
@yysung.bsky.social is presenting: GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration
x.com/YooYeonSung1...
(Short version: quiz bowl, a dumb trivia game, shows humans' calibration > LLMs'.)
@yysung.bsky.social is presenting: GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration
x.com/YooYeonSung1...
(Short version: quiz bowl, a dumb trivia game, shows humans' calibration > LLMs'.)
Yoo Yeon Sung@ACL2025 on X: "I’ll be presenting this work in Room 1.62 today! If you're curious about how calibration errors in LLMs can be measured through human calibration, come find me and @enfleisig! 📍Oral Session 3 - HC: Human-centered NLP 📅Monday, July 28@ 2PM" / X
I’ll be presenting this work in Room 1.62 today! If you're curious about how calibration errors in LLMs can be measured through human calibration, come find me and @enfleisig! 📍Oral Session 3 - HC: Human-centered NLP 📅Monday, July 28@ 2PM
x.com
July 28, 2025 at 8:35 AM
In the second oral paper (14:22 PM, Room 1.62),
@yysung.bsky.social is presenting: GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration
x.com/YooYeonSung1...
(Short version: quiz bowl, a dumb trivia game, shows humans' calibration > LLMs'.)
@yysung.bsky.social is presenting: GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration
x.com/YooYeonSung1...
(Short version: quiz bowl, a dumb trivia game, shows humans' calibration > LLMs'.)
In the first poster session (11AM Monday, Hall X4/X5),
@nbalepur.bsky.social is presenting: Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas
x.com/NishantBalep...
@nbalepur.bsky.social is presenting: Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas
x.com/NishantBalep...
Nishant is ill-prepared for ACL2025 on X: "While personalization is great, it's not perfect. We found our strategy could easily let users jailbreak the model 😭 With some extra safeguards (e.g. refusal training), we think inferred personas could become a promising way to boost personalization in post-training recipes! 🧑🍳 https://t.co/Fpu1Bl34fI" / X
While personalization is great, it's not perfect. We found our strategy could easily let users jailbreak the model 😭 With some extra safeguards (e.g. refusal training), we think inferred personas could become a promising way to boost personalization in post-training recipes! 🧑🍳 https://t.co/Fpu1Bl34fI
x.com
July 28, 2025 at 8:35 AM
In the first poster session (11AM Monday, Hall X4/X5),
@nbalepur.bsky.social is presenting: Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas
x.com/NishantBalep...
@nbalepur.bsky.social is presenting: Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas
x.com/NishantBalep...
And you can signup for online mirror (June 21, 12:00 EST) here:
docs.google.com/forms/d/e/1F...
[Signup deadline: June 18 Anywhere on Earth]
docs.google.com/forms/d/e/1F...
[Signup deadline: June 18 Anywhere on Earth]
2025 QANTA Player Signup
Sign up for the human competition for our 2025 QANTA event.
More information:
https://sites.google.com/view/qanta/2025-competition/2025-human-teams
docs.google.com
June 17, 2025 at 3:35 PM
And you can signup for online mirror (June 21, 12:00 EST) here:
docs.google.com/forms/d/e/1F...
[Signup deadline: June 18 Anywhere on Earth]
docs.google.com/forms/d/e/1F...
[Signup deadline: June 18 Anywhere on Earth]
If this sounds like fun, we have more information on the setup here:
sites.google.com/view/qanta/2...
sites.google.com/view/qanta/2...
QANTA Project - 2025 Human Teams
How this Works
It's always difficult to try something new, but we think this will be fun!
Short Version: You'll play multiple rounds of tossup-bonus quiz bowl with computer teammates. The computer ca...
sites.google.com
June 17, 2025 at 3:35 PM
If this sounds like fun, we have more information on the setup here:
sites.google.com/view/qanta/2...
sites.google.com/view/qanta/2...
There are more reflections here:
hsquizbowl.org/forums/viewt...
hsquizbowl.org/forums/viewt...
Human-Computer Cooperation Tournament (College Park: June 14, Online: June 21) - The Quizbowl Resource Center
Sponsored by the Partnership for Academic Competition Excellence
hsquizbowl.org
June 17, 2025 at 3:35 PM
There are more reflections here:
hsquizbowl.org/forums/viewt...
hsquizbowl.org/forums/viewt...
Sara’s Crias went 4-2 to win the tournament (and $150 dollars). Noah Sheidlower’s music packet was the most difficult for computers, and Jame Carlson’s Spatial Reasoning was the fan favorite. We’ll announce writer and computer prizes after our online mirror. (And also post the packets.)
June 17, 2025 at 3:35 PM
Sara’s Crias went 4-2 to win the tournament (and $150 dollars). Noah Sheidlower’s music packet was the most difficult for computers, and Jame Carlson’s Spatial Reasoning was the fan favorite. We’ll announce writer and computer prizes after our online mirror. (And also post the packets.)
And get more information here:
sites.google.com/view/qanta/2...
sites.google.com/view/qanta/2...
QANTA Project - 2025 Human Teams
How this Works
It's always difficult to try something new, but we think this will be fun!
Short Version: You'll play multiple rounds of tossup-bonus quiz bowl with computer teammates. The computer ca...
sites.google.com
June 10, 2025 at 4:23 PM
And get more information here:
sites.google.com/view/qanta/2...
sites.google.com/view/qanta/2...
Sign up to play here:
docs.google.com/forms/d/e/1F...
docs.google.com/forms/d/e/1F...
2025 QANTA Player Signup
Sign up for the in-person competition for our 2025 QANTA event.
More information:
https://sites.google.com/view/qanta/2025-competition/2025-human-teams
docs.google.com
June 10, 2025 at 4:23 PM
Sign up to play here:
docs.google.com/forms/d/e/1F...
docs.google.com/forms/d/e/1F...