Werner Geyer
@wernergeyer.bsky.social
Chief Scientist Human-Center Trustworthy AI @ IBM Research. Interested in Human+AI Interaction & AI-Assisted Productivity. Opinions are my own! https://wernergeyer.com
Pinned
Werner Geyer
@wernergeyer.bsky.social
· Jun 16
LLM-as-a-Judge Without the Headaches: EvalAssist Brings Structure and Simplicity to the Chaos of LLM Output Review | AI Alliance
Evaluating AI model outputs at scale is a major challenge for teams using LLMs, especially when assessing nuanced qualities like politeness, fairness, and tone that traditional benchmarks miss. IBM Re...
thealliance.ai
📣 Today we open-sourced EvalAssist, a web-based tool that makes it super easy to develop criteria for llm judges. You can run this now locally and then scale up with notebooks using Unitxt. Check out the AI Alliance article to get the scoop:
thealliance.ai/blog/llm-as-...
thealliance.ai/blog/llm-as-...
Reposted by Werner Geyer
Picard technology tip: Sometimes your chief engineer can build new systems that are better than your existing enterprise software.
October 19, 2025 at 8:04 PM
Picard technology tip: Sometimes your chief engineer can build new systems that are better than your existing enterprise software.
Reposted by Werner Geyer
❣️ Shout out to my amazing co-authors:
Rachel Ostrand, @wernergeyer.bsky.social , @keerthi166.bsky.social, Dennis Wei, and Justin Weisz!
If you'll be at AIES, I would love to connect and chat more about our work! 🙌
Rachel Ostrand, @wernergeyer.bsky.social , @keerthi166.bsky.social, Dennis Wei, and Justin Weisz!
If you'll be at AIES, I would love to connect and chat more about our work! 🙌
October 16, 2025 at 11:01 AM
❣️ Shout out to my amazing co-authors:
Rachel Ostrand, @wernergeyer.bsky.social , @keerthi166.bsky.social, Dennis Wei, and Justin Weisz!
If you'll be at AIES, I would love to connect and chat more about our work! 🙌
Rachel Ostrand, @wernergeyer.bsky.social , @keerthi166.bsky.social, Dennis Wei, and Justin Weisz!
If you'll be at AIES, I would love to connect and chat more about our work! 🙌
Reposted by Werner Geyer
Picard management tip: Even without game-changing results, experimentation is time well spent.
September 26, 2025 at 8:49 PM
Picard management tip: Even without game-changing results, experimentation is time well spent.
🚀 Excited to share some updates from EvalAssist, the open-source LLM-as-a-Judge framework we released a few months ago! 🧵
September 25, 2025 at 5:56 PM
🚀 Excited to share some updates from EvalAssist, the open-source LLM-as-a-Judge framework we released a few months ago! 🧵
We've just extended the IUI Workshop deadline by one week to August 29.
Looking forward to your contributions!
Looking forward to your contributions!
📢 Call for Workshop & Tutorial Proposals 📢
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
August 21, 2025 at 1:55 PM
We've just extended the IUI Workshop deadline by one week to August 29.
Looking forward to your contributions!
Looking forward to your contributions!
Getting ready! Come visit us at the IBM booth @acl to learn about our latest Research. We have a number of super interesting demos lined up. research.ibm.com/events/acl-2...
July 28, 2025 at 7:44 AM
Getting ready! Come visit us at the IBM booth @acl to learn about our latest Research. We have a number of super interesting demos lined up. research.ibm.com/events/acl-2...
Reposted by Werner Geyer
We’re growing and going global! 🌍
CHIWORK 2025 is shaping up to be our biggest and most diverse edition yet. Thanks to everyone who submitted, reviewed, and supported us 💙
Can’t wait to see you in Amsterdam!
🔗 chiwork.org
#CHIWORK2025 #HCI #FutureOfWork
CHIWORK 2025 is shaping up to be our biggest and most diverse edition yet. Thanks to everyone who submitted, reviewed, and supported us 💙
Can’t wait to see you in Amsterdam!
🔗 chiwork.org
#CHIWORK2025 #HCI #FutureOfWork
April 4, 2025 at 10:59 AM
We’re growing and going global! 🌍
CHIWORK 2025 is shaping up to be our biggest and most diverse edition yet. Thanks to everyone who submitted, reviewed, and supported us 💙
Can’t wait to see you in Amsterdam!
🔗 chiwork.org
#CHIWORK2025 #HCI #FutureOfWork
CHIWORK 2025 is shaping up to be our biggest and most diverse edition yet. Thanks to everyone who submitted, reviewed, and supported us 💙
Can’t wait to see you in Amsterdam!
🔗 chiwork.org
#CHIWORK2025 #HCI #FutureOfWork
Reposted by Werner Geyer
📢 Call for Workshop & Tutorial Proposals 📢
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
June 9, 2025 at 2:39 PM
📢 Call for Workshop & Tutorial Proposals 📢
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026.
iui.hosting.acm.org/2026/call-fo...
#CallForProposals #IUI2026 #HCI #AI
📣 Today we open-sourced EvalAssist, a web-based tool that makes it super easy to develop criteria for llm judges. You can run this now locally and then scale up with notebooks using Unitxt. Check out the AI Alliance article to get the scoop:
thealliance.ai/blog/llm-as-...
thealliance.ai/blog/llm-as-...
LLM-as-a-Judge Without the Headaches: EvalAssist Brings Structure and Simplicity to the Chaos of LLM Output Review | AI Alliance
Evaluating AI model outputs at scale is a major challenge for teams using LLMs, especially when assessing nuanced qualities like politeness, fairness, and tone that traditional benchmarks miss. IBM Re...
thealliance.ai
June 16, 2025 at 3:38 PM
📣 Today we open-sourced EvalAssist, a web-based tool that makes it super easy to develop criteria for llm judges. You can run this now locally and then scale up with notebooks using Unitxt. Check out the AI Alliance article to get the scoop:
thealliance.ai/blog/llm-as-...
thealliance.ai/blog/llm-as-...
Reposted by Werner Geyer
📣 Call for Workshop & Tutorial Proposals 📣 #IUI2026 is looking forward to your contribution! Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026. 🚨 Proposal Deadlines: Aug 22 (Workshops) and Oct 17 (Tutorials)🚨 iui.hosting.acm.org/2026/call-fo...
Call for Workshop & Tutorial Proposals | IUI
iui.hosting.acm.org
June 5, 2025 at 4:02 PM
📣 Call for Workshop & Tutorial Proposals 📣 #IUI2026 is looking forward to your contribution! Bring your ideas and discuss them with fellow researchers in Paphos, Cyprus, from March 22-26, 2026. 🚨 Proposal Deadlines: Aug 22 (Workshops) and Oct 17 (Tutorials)🚨 iui.hosting.acm.org/2026/call-fo...
📣 IUI 2026 Call for Workshops and Tutorials is live 📣
iui.acm.org/2026/call-fo...
Note that this year, submissions will be due August 22 earlier than previous years. Pls. spread the word! We had a fantastic workshop program in 2025 and I'm looking forward to an even better one in 2026 in Cyprus.
iui.acm.org/2026/call-fo...
Note that this year, submissions will be due August 22 earlier than previous years. Pls. spread the word! We had a fantastic workshop program in 2025 and I'm looking forward to an even better one in 2026 in Cyprus.
Call for Workshop & Tutorial Proposals | IUI
iui.acm.org
June 5, 2025 at 3:09 PM
📣 IUI 2026 Call for Workshops and Tutorials is live 📣
iui.acm.org/2026/call-fo...
Note that this year, submissions will be due August 22 earlier than previous years. Pls. spread the word! We had a fantastic workshop program in 2025 and I'm looking forward to an even better one in 2026 in Cyprus.
iui.acm.org/2026/call-fo...
Note that this year, submissions will be due August 22 earlier than previous years. Pls. spread the word! We had a fantastic workshop program in 2025 and I'm looking forward to an even better one in 2026 in Cyprus.
We just published a summary the 6th workshop on Human-AI Co-Creation with Generative Models at IUI 2025 in March. This year's special topic, of course, AI agents and agency. Two of our sessions covered this topic and we had an exciting panel discussion. Check it out! medium.com/human-center...
HAI-GEN 2025: 6th Workshop on Human-AI Co-Creation with Generative Models
by Osnat Mokryn (University of Haifa, IL), Orit Shaer (Wellesley College, US), Werner Geyer (IBM Research, US), Mary Lou Maher (Computing…
medium.com
May 6, 2025 at 6:35 PM
We just published a summary the 6th workshop on Human-AI Co-Creation with Generative Models at IUI 2025 in March. This year's special topic, of course, AI agents and agency. Two of our sessions covered this topic and we had an exciting panel discussion. Check it out! medium.com/human-center...
Great work from our team @ IBM Research
🤖 ✏️ There is a better way to explain how you used AI in your {research paper, college essay, blog posts, …}. Check out our new AI Attribution Toolkit and look for us at #CHI2025!
aiattribution.github.io
dl.acm.org/doi/full/10....
aiattribution.github.io
dl.acm.org/doi/full/10....
AI Attribution Toolkit
An attribution statement identifies not only the presence of AI involvement, but also how AI was used. This approach makes important distinctions between different types and amounts of AI…
aiattribution.github.io
April 29, 2025 at 11:26 PM
Great work from our team @ IBM Research
Reposted by Werner Geyer
A summary of decolonial AI alignment in the Human-Centered AI publication on Medium. Thanks to @jweisz3.bsky.social for asking me to write it, and for editing the piece. medium.com/human-center...
Decolonial AI Alignment
by Kush Varshney (IBM Research, US)
medium.com
April 8, 2025 at 3:12 PM
A summary of decolonial AI alignment in the Human-Centered AI publication on Medium. Thanks to @jweisz3.bsky.social for asking me to write it, and for editing the piece. medium.com/human-center...
Reposted by Werner Geyer
I'm on the IBM Mixture of Experts podcast wearing a safety vest. We talk about all the new things in AI this week. I also connect to older work by IBM Fellows Irene Greif, Bob Dennard, Rolf Landauer, and Charlie Bennett and to Mauro Martino's new AI-generated film. www.youtube.com/watch?v=CgqH...
DeepSeek-V3-0324, Gemini Canvas and GPT-4o image generation
YouTube video by IBM Technology
www.youtube.com
March 28, 2025 at 1:10 PM
I'm on the IBM Mixture of Experts podcast wearing a safety vest. We talk about all the new things in AI this week. I also connect to older work by IBM Fellows Irene Greif, Bob Dennard, Rolf Landauer, and Charlie Bennett and to Mauro Martino's new AI-generated film. www.youtube.com/watch?v=CgqH...
Reposted by Werner Geyer
Granite Guardian tops a new benchmark! research.ibm.com/blog/granite...
Granite Guardian tops third-party AI benchmark
IBM’s collection of LLM guardrail models take six of the top 10 spots on the new GuardBench leaderboard.
research.ibm.com
April 9, 2025 at 7:51 PM
Granite Guardian tops a new benchmark! research.ibm.com/blog/granite...
And the final product 😋
April 1, 2025 at 3:40 PM
And the final product 😋
Asparagus time in Germany. This is an automated peeling machine. No AI 😀
March 31, 2025 at 12:22 PM
Asparagus time in Germany. This is an automated peeling machine. No AI 😀
All set up for demo time at IUI. We are showing a tool for GenAU-assisted hypotheses exploration. dl.acm.org/doi/10.1145/...
March 26, 2025 at 3:39 PM
All set up for demo time at IUI. We are showing a tool for GenAU-assisted hypotheses exploration. dl.acm.org/doi/10.1145/...
IBM Research on their way to IUI
March 22, 2025 at 3:53 PM
IBM Research on their way to IUI
Ah, and now there is a cool name for it :) www.zdnet.com/article/what...
Is there already a CHI paper about it? :)
Is there already a CHI paper about it? :)
What is AI vibe coding? It's all the rage but it's not for everyone - here's why
Caution: Experience required. Vibe coding feels like magic, until your AI assistant starts overwriting your work.
www.zdnet.com
March 18, 2025 at 4:44 PM
Ah, and now there is a cool name for it :) www.zdnet.com/article/what...
Is there already a CHI paper about it? :)
Is there already a CHI paper about it? :)
We have two amazing keynotes this year at HAI-GEN 2025 to challenge our thinking on co-creative systems from an interaction perspective.
Hope to cu at IUI this year!
hai-gen.github.io/2025/program/
Hope to cu at IUI this year!
hai-gen.github.io/2025/program/
February 11, 2025 at 3:38 PM
We have two amazing keynotes this year at HAI-GEN 2025 to challenge our thinking on co-creative systems from an interaction perspective.
Hope to cu at IUI this year!
hai-gen.github.io/2025/program/
Hope to cu at IUI this year!
hai-gen.github.io/2025/program/
Reposted by Werner Geyer
📣 The #CSCW2026 deadline (@acm-cscw.bsky.social) has been posted. Big change this year. There is **only one deadline** for 2026 and it is May 13, 2025. 📣
Please spread the word!
#CSCW #CHI #HCI #socialcomputing
cscw.acm.org/2025/index.p...
Please spread the word!
#CSCW #CHI #HCI #socialcomputing
cscw.acm.org/2025/index.p...
CALL FOR PAPERS – CSCW 2025
cscw.acm.org
January 31, 2025 at 1:51 PM
📣 The #CSCW2026 deadline (@acm-cscw.bsky.social) has been posted. Big change this year. There is **only one deadline** for 2026 and it is May 13, 2025. 📣
Please spread the word!
#CSCW #CHI #HCI #socialcomputing
cscw.acm.org/2025/index.p...
Please spread the word!
#CSCW #CHI #HCI #socialcomputing
cscw.acm.org/2025/index.p...
Looking forward seeing your papers!!!
We just extended the submission deadline for the HAI-GEN workshop at #IUI2025. Please submit your work by Fri Jan 24, AoE!
hai-gen.github.io/2025/
@ossimokryn.bsky.social, @wernergeyer.bsky.social
hai-gen.github.io/2025/
@ossimokryn.bsky.social, @wernergeyer.bsky.social
HAI-GEN Workshop at IUI 2025
6th Workshop on Human-AI Co-Creation with Generative Models
hai-gen.github.io
January 23, 2025 at 1:19 PM
Looking forward seeing your papers!!!
Reposted by Werner Geyer
"IBM has equipped the Granite Guardian 3.1 models with the ability to detect hallucinations in AI agent workflows. This feature provides oversight of an AI agent completing a task, monitoring for fabricated information or incorrect function calls." technologymagazine.com/articles/the...
The Key to How IBM's Granite 3.1 is Advancing Enterprise AI
IBM’s new Granite 3.1 addresses key enterprise needs, including expanded context handling, multilingual support, new tools and AI agent development
technologymagazine.com
December 20, 2024 at 9:27 PM
"IBM has equipped the Granite Guardian 3.1 models with the ability to detect hallucinations in AI agent workflows. This feature provides oversight of an AI agent completing a task, monitoring for fabricated information or incorrect function calls." technologymagazine.com/articles/the...