Stella Li
@stellali.bsky.social
PhD student @uwnlp.bsky.social @uwcse.bsky.social | visiting researcher @MetaAI | previously @jhuclsp.bsky.social
https://stellalisy.com
https://stellalisy.com
Reposted by Stella Li
Day 1 (Tue Oct 7) 4:30-6:30pm, Poster Session 2
Poster #77: ALFA: Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning; led by
@stellali.bsky.social & @jiminmun.bsky.social
Poster #77: ALFA: Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning; led by
@stellali.bsky.social & @jiminmun.bsky.social
October 6, 2025 at 2:51 PM
Day 1 (Tue Oct 7) 4:30-6:30pm, Poster Session 2
Poster #77: ALFA: Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning; led by
@stellali.bsky.social & @jiminmun.bsky.social
Poster #77: ALFA: Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning; led by
@stellali.bsky.social & @jiminmun.bsky.social
This project was done as part of the Meta FAIR AIM mentorship program. Special thanks to my amazing collaborators and awesome mentors @melaniesclar.bsky.social @jcqln_h @hunterjlang @AnsongNi @andrew_e_cohen @jacoby_xu @chan_young_park @tsvetshop.bsky.social @asli-celikyilmaz.bsky.social 🫶🏻💙
July 22, 2025 at 2:59 PM
This project was done as part of the Meta FAIR AIM mentorship program. Special thanks to my amazing collaborators and awesome mentors @melaniesclar.bsky.social @jcqln_h @hunterjlang @AnsongNi @andrew_e_cohen @jacoby_xu @chan_young_park @tsvetshop.bsky.social @asli-celikyilmaz.bsky.social 🫶🏻💙
✨PrefPalette🎨 bridges cognitive science, social psychology, and AI for explainable preference modeling✨
📖Paper: arxiv.org/abs/2507.13541
💻Code: github.com/stellalisy/P...
Join us in shaping interpretable AI that you can trust and control🚀Feedback welcome!
#AI #Transparency
📖Paper: arxiv.org/abs/2507.13541
💻Code: github.com/stellalisy/P...
Join us in shaping interpretable AI that you can trust and control🚀Feedback welcome!
#AI #Transparency
PrefPalette: Personalized Preference Modeling with Latent Attributes
Personalizing AI systems requires understanding not just what users prefer, but the reasons that underlie those preferences - yet current preference models typically treat human judgment as a black bo...
arxiv.org
July 22, 2025 at 2:59 PM
✨PrefPalette🎨 bridges cognitive science, social psychology, and AI for explainable preference modeling✨
📖Paper: arxiv.org/abs/2507.13541
💻Code: github.com/stellalisy/P...
Join us in shaping interpretable AI that you can trust and control🚀Feedback welcome!
#AI #Transparency
📖Paper: arxiv.org/abs/2507.13541
💻Code: github.com/stellalisy/P...
Join us in shaping interpretable AI that you can trust and control🚀Feedback welcome!
#AI #Transparency
🌍Bonus: PrefPalette🎨 is a computational social science goldmine!
📊 Quantify community values at scale
📈 Track how norms evolve over time
🔍 Understand group psychology
📋 Move beyond surveys to revealed preferences
📊 Quantify community values at scale
📈 Track how norms evolve over time
🔍 Understand group psychology
📋 Move beyond surveys to revealed preferences
July 22, 2025 at 2:59 PM
🌍Bonus: PrefPalette🎨 is a computational social science goldmine!
📊 Quantify community values at scale
📈 Track how norms evolve over time
🔍 Understand group psychology
📋 Move beyond surveys to revealed preferences
📊 Quantify community values at scale
📈 Track how norms evolve over time
🔍 Understand group psychology
📋 Move beyond surveys to revealed preferences
💡Potential real-world applications:
🛡️Smart content moderation—explains why content is flagged/decisions are made
🎯Interpretable LM alignment—revealing prominent attributes
⚙️Controllable personalization—giving user agency to personalize select attributes
🛡️Smart content moderation—explains why content is flagged/decisions are made
🎯Interpretable LM alignment—revealing prominent attributes
⚙️Controllable personalization—giving user agency to personalize select attributes
July 22, 2025 at 2:59 PM
💡Potential real-world applications:
🛡️Smart content moderation—explains why content is flagged/decisions are made
🎯Interpretable LM alignment—revealing prominent attributes
⚙️Controllable personalization—giving user agency to personalize select attributes
🛡️Smart content moderation—explains why content is flagged/decisions are made
🎯Interpretable LM alignment—revealing prominent attributes
⚙️Controllable personalization—giving user agency to personalize select attributes
🔍More importantly‼️we can see WHY preferences differ:
r/AskHistorians:📚values verbosity
r/RoastMe:💥values directness
r/confession:❤️values empathy
We visualize each group’s unique preference decisions—no more one-size-fits-all. Understand your audience at a glance🏷️
r/AskHistorians:📚values verbosity
r/RoastMe:💥values directness
r/confession:❤️values empathy
We visualize each group’s unique preference decisions—no more one-size-fits-all. Understand your audience at a glance🏷️
July 22, 2025 at 2:59 PM
🔍More importantly‼️we can see WHY preferences differ:
r/AskHistorians:📚values verbosity
r/RoastMe:💥values directness
r/confession:❤️values empathy
We visualize each group’s unique preference decisions—no more one-size-fits-all. Understand your audience at a glance🏷️
r/AskHistorians:📚values verbosity
r/RoastMe:💥values directness
r/confession:❤️values empathy
We visualize each group’s unique preference decisions—no more one-size-fits-all. Understand your audience at a glance🏷️
🏆Results across 45 Reddit communities:
📈Performance boost: +46.6% vs GPT-4o
💪Outperforms other training-based baselines w/ statistical significance
🕰️Robust to temporal shifts—trained pref models can be used out-of-the box!
📈Performance boost: +46.6% vs GPT-4o
💪Outperforms other training-based baselines w/ statistical significance
🕰️Robust to temporal shifts—trained pref models can be used out-of-the box!
July 22, 2025 at 2:59 PM
🏆Results across 45 Reddit communities:
📈Performance boost: +46.6% vs GPT-4o
💪Outperforms other training-based baselines w/ statistical significance
🕰️Robust to temporal shifts—trained pref models can be used out-of-the box!
📈Performance boost: +46.6% vs GPT-4o
💪Outperforms other training-based baselines w/ statistical significance
🕰️Robust to temporal shifts—trained pref models can be used out-of-the box!
⚙️How it works (pt.2)
1: 🎛️Train compact, efficient detectors for every attribute
2: 🎯Learn community-specific attribute weights during preference training
3: 🔧Add attribute embeddings to preference model for accurate & explainable predictions
1: 🎛️Train compact, efficient detectors for every attribute
2: 🎯Learn community-specific attribute weights during preference training
3: 🔧Add attribute embeddings to preference model for accurate & explainable predictions
July 22, 2025 at 2:59 PM
⚙️How it works (pt.2)
1: 🎛️Train compact, efficient detectors for every attribute
2: 🎯Learn community-specific attribute weights during preference training
3: 🔧Add attribute embeddings to preference model for accurate & explainable predictions
1: 🎛️Train compact, efficient detectors for every attribute
2: 🎯Learn community-specific attribute weights during preference training
3: 🔧Add attribute embeddings to preference model for accurate & explainable predictions
⚙️How it works (prep stage)
📜Define 19 sociolinguistics & cultural attributes from literature
🏭Novel preference data generation pipeline to isolate attributes
Our data gen pipeline generates pairwise data on *any* decomposed dimension, w/ applications beyond preference modeling
📜Define 19 sociolinguistics & cultural attributes from literature
🏭Novel preference data generation pipeline to isolate attributes
Our data gen pipeline generates pairwise data on *any* decomposed dimension, w/ applications beyond preference modeling
July 22, 2025 at 2:59 PM
⚙️How it works (prep stage)
📜Define 19 sociolinguistics & cultural attributes from literature
🏭Novel preference data generation pipeline to isolate attributes
Our data gen pipeline generates pairwise data on *any* decomposed dimension, w/ applications beyond preference modeling
📜Define 19 sociolinguistics & cultural attributes from literature
🏭Novel preference data generation pipeline to isolate attributes
Our data gen pipeline generates pairwise data on *any* decomposed dimension, w/ applications beyond preference modeling
Meet PrefPalette🎨! Our approach:
🔍⚖️models preferences w/ 19 attribute detectors and dynamic, context-aware weights
🕶️👍uses unobtrusive signals from Reddit to avoid response bias
🧠mirrors attribute-mediated human judgment—so you know not just what it predicts, but *why*🧐
🔍⚖️models preferences w/ 19 attribute detectors and dynamic, context-aware weights
🕶️👍uses unobtrusive signals from Reddit to avoid response bias
🧠mirrors attribute-mediated human judgment—so you know not just what it predicts, but *why*🧐
July 22, 2025 at 2:59 PM
Meet PrefPalette🎨! Our approach:
🔍⚖️models preferences w/ 19 attribute detectors and dynamic, context-aware weights
🕶️👍uses unobtrusive signals from Reddit to avoid response bias
🧠mirrors attribute-mediated human judgment—so you know not just what it predicts, but *why*🧐
🔍⚖️models preferences w/ 19 attribute detectors and dynamic, context-aware weights
🕶️👍uses unobtrusive signals from Reddit to avoid response bias
🧠mirrors attribute-mediated human judgment—so you know not just what it predicts, but *why*🧐
🔬Cognitive science reveals how humans break choices into attributes, e.g.:
😂 Humor
❤️ Empathy
💬 Conformity
...then weight them based on context (e.g. comedy vs counseling).
These traits shape every decision, from product picks to conversation tone. Your mind is a colorful palette🎨
😂 Humor
❤️ Empathy
💬 Conformity
...then weight them based on context (e.g. comedy vs counseling).
These traits shape every decision, from product picks to conversation tone. Your mind is a colorful palette🎨
July 22, 2025 at 2:59 PM
🔬Cognitive science reveals how humans break choices into attributes, e.g.:
😂 Humor
❤️ Empathy
💬 Conformity
...then weight them based on context (e.g. comedy vs counseling).
These traits shape every decision, from product picks to conversation tone. Your mind is a colorful palette🎨
😂 Humor
❤️ Empathy
💬 Conformity
...then weight them based on context (e.g. comedy vs counseling).
These traits shape every decision, from product picks to conversation tone. Your mind is a colorful palette🎨
🚨Current preference models only output a reward/score:
❌No transparency in decision-making
❌Personalization breaks easily, one-size-fits-all scores
❌Use explicit annotations (response bias)
They can’t adapt to individual tastes, can’t debug errors, and fail to build trust🙅
❌No transparency in decision-making
❌Personalization breaks easily, one-size-fits-all scores
❌Use explicit annotations (response bias)
They can’t adapt to individual tastes, can’t debug errors, and fail to build trust🙅
July 22, 2025 at 2:59 PM
🚨Current preference models only output a reward/score:
❌No transparency in decision-making
❌Personalization breaks easily, one-size-fits-all scores
❌Use explicit annotations (response bias)
They can’t adapt to individual tastes, can’t debug errors, and fail to build trust🙅
❌No transparency in decision-making
❌Personalization breaks easily, one-size-fits-all scores
❌Use explicit annotations (response bias)
They can’t adapt to individual tastes, can’t debug errors, and fail to build trust🙅
This work was jointly done with the amazing @jiminmun.bsky.social !
And huge shout out to our awesome collaborators and mentors faebrahman.bsky.social, Jonathan Ilgen, Yulia (tsvetshop.bsky.social) and maartensap.bsky.social 🩵🥰
And huge shout out to our awesome collaborators and mentors faebrahman.bsky.social, Jonathan Ilgen, Yulia (tsvetshop.bsky.social) and maartensap.bsky.social 🩵🥰
Faeze Brahman (@faebrahman.bsky.social)
faebrahman.bsky.social
February 21, 2025 at 4:13 PM
This work was jointly done with the amazing @jiminmun.bsky.social !
And huge shout out to our awesome collaborators and mentors faebrahman.bsky.social, Jonathan Ilgen, Yulia (tsvetshop.bsky.social) and maartensap.bsky.social 🩵🥰
And huge shout out to our awesome collaborators and mentors faebrahman.bsky.social, Jonathan Ilgen, Yulia (tsvetshop.bsky.social) and maartensap.bsky.social 🩵🥰
ALFA is open-source! There are many more analyses in the paper, check them out!📖
🔗 Paper: arxiv.org/abs/2502.14860
💻 Code: github.com/stellalisy/ALFA
🤖 Data: tinyurl.com/MedicAskDocsData
Join us in the effort to make LLMs better at question asking! 🚀
#Healthcare #NLProc #AI4Science
🔗 Paper: arxiv.org/abs/2502.14860
💻 Code: github.com/stellalisy/ALFA
🤖 Data: tinyurl.com/MedicAskDocsData
Join us in the effort to make LLMs better at question asking! 🚀
#Healthcare #NLProc #AI4Science
Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning
Large language models (LLMs) often fail to ask effective questions under uncertainty, making them unreliable in domains where proactive information-gathering is essential for decisionmaking. We presen...
arxiv.org
February 21, 2025 at 4:12 PM
ALFA is open-source! There are many more analyses in the paper, check them out!📖
🔗 Paper: arxiv.org/abs/2502.14860
💻 Code: github.com/stellalisy/ALFA
🤖 Data: tinyurl.com/MedicAskDocsData
Join us in the effort to make LLMs better at question asking! 🚀
#Healthcare #NLProc #AI4Science
🔗 Paper: arxiv.org/abs/2502.14860
💻 Code: github.com/stellalisy/ALFA
🤖 Data: tinyurl.com/MedicAskDocsData
Join us in the effort to make LLMs better at question asking! 🚀
#Healthcare #NLProc #AI4Science
Why this matters for AI safety & reliability: 🛡️
Better information gathering = Better decisions✅
Proactive questioning = Fewer blind spots🧐
Structured attributes = More controllable AI🤖
Interactive systems = More natural AI assistants🫶🏻
Better information gathering = Better decisions✅
Proactive questioning = Fewer blind spots🧐
Structured attributes = More controllable AI🤖
Interactive systems = More natural AI assistants🫶🏻
February 21, 2025 at 4:09 PM
Why this matters for AI safety & reliability: 🛡️
Better information gathering = Better decisions✅
Proactive questioning = Fewer blind spots🧐
Structured attributes = More controllable AI🤖
Interactive systems = More natural AI assistants🫶🏻
Better information gathering = Better decisions✅
Proactive questioning = Fewer blind spots🧐
Structured attributes = More controllable AI🤖
Interactive systems = More natural AI assistants🫶🏻
ALFA isn't just for medicine! The framework could be adapted to ANY field where proactive information gathering matters:
Legal consultation ⚖️
Financial advising 💰
Educational tutoring 📚
Investigative journalism 🕵️
Anywhere an AI needs to ask (not just answer), you should try ALFA out!🌟
Legal consultation ⚖️
Financial advising 💰
Educational tutoring 📚
Investigative journalism 🕵️
Anywhere an AI needs to ask (not just answer), you should try ALFA out!🌟
February 21, 2025 at 4:08 PM
ALFA isn't just for medicine! The framework could be adapted to ANY field where proactive information gathering matters:
Legal consultation ⚖️
Financial advising 💰
Educational tutoring 📚
Investigative journalism 🕵️
Anywhere an AI needs to ask (not just answer), you should try ALFA out!🌟
Legal consultation ⚖️
Financial advising 💰
Educational tutoring 📚
Investigative journalism 🕵️
Anywhere an AI needs to ask (not just answer), you should try ALFA out!🌟
🌟 Impressive Generalization!
ALFA-trained models maintain strong performance even on completely new interactive medical tasks (MediQ-MedQA).
highlighting ALFA’s potential for broader applicability in real-world clinical scenarios‼️
ALFA-trained models maintain strong performance even on completely new interactive medical tasks (MediQ-MedQA).
highlighting ALFA’s potential for broader applicability in real-world clinical scenarios‼️
February 21, 2025 at 4:08 PM
🌟 Impressive Generalization!
ALFA-trained models maintain strong performance even on completely new interactive medical tasks (MediQ-MedQA).
highlighting ALFA’s potential for broader applicability in real-world clinical scenarios‼️
ALFA-trained models maintain strong performance even on completely new interactive medical tasks (MediQ-MedQA).
highlighting ALFA’s potential for broader applicability in real-world clinical scenarios‼️
🔬 Key Finding #2: Every Attribute Matters!
Removing any single attribute hurts performance‼️
Grouping general (clarify, focus, answerability) vs. clinical (medical accuracy, diagnostic relevance, avoiding DDX bias) attributes leads to drastically different outputs👩⚕️
Check out some cool examples!👇
Removing any single attribute hurts performance‼️
Grouping general (clarify, focus, answerability) vs. clinical (medical accuracy, diagnostic relevance, avoiding DDX bias) attributes leads to drastically different outputs👩⚕️
Check out some cool examples!👇
February 21, 2025 at 4:07 PM
🔬 Key Finding #2: Every Attribute Matters!
Removing any single attribute hurts performance‼️
Grouping general (clarify, focus, answerability) vs. clinical (medical accuracy, diagnostic relevance, avoiding DDX bias) attributes leads to drastically different outputs👩⚕️
Check out some cool examples!👇
Removing any single attribute hurts performance‼️
Grouping general (clarify, focus, answerability) vs. clinical (medical accuracy, diagnostic relevance, avoiding DDX bias) attributes leads to drastically different outputs👩⚕️
Check out some cool examples!👇
🔬 Key Finding #1: Preference Learning > Supervised Learning
Is it just good synthetic data❓ No❗️
Simply showing good examples isn't enough! Models need to learn directional differences between good and bad questions.
(but only SFT no DPO also doesn't work!)
Is it just good synthetic data❓ No❗️
Simply showing good examples isn't enough! Models need to learn directional differences between good and bad questions.
(but only SFT no DPO also doesn't work!)
February 21, 2025 at 4:05 PM
🔬 Key Finding #1: Preference Learning > Supervised Learning
Is it just good synthetic data❓ No❗️
Simply showing good examples isn't enough! Models need to learn directional differences between good and bad questions.
(but only SFT no DPO also doesn't work!)
Is it just good synthetic data❓ No❗️
Simply showing good examples isn't enough! Models need to learn directional differences between good and bad questions.
(but only SFT no DPO also doesn't work!)
Results show ALFA’s strengths🚀
ALFA-aligned models achieve:
⭐️56.6% reduction in diagnostic errors🦾
⭐️64.4% win rate in question quality✅
⭐️Strong generalization.
in comparison with baseline SoTA instruction-tuned LLMs.
ALFA-aligned models achieve:
⭐️56.6% reduction in diagnostic errors🦾
⭐️64.4% win rate in question quality✅
⭐️Strong generalization.
in comparison with baseline SoTA instruction-tuned LLMs.
February 21, 2025 at 4:04 PM
Results show ALFA’s strengths🚀
ALFA-aligned models achieve:
⭐️56.6% reduction in diagnostic errors🦾
⭐️64.4% win rate in question quality✅
⭐️Strong generalization.
in comparison with baseline SoTA instruction-tuned LLMs.
ALFA-aligned models achieve:
⭐️56.6% reduction in diagnostic errors🦾
⭐️64.4% win rate in question quality✅
⭐️Strong generalization.
in comparison with baseline SoTA instruction-tuned LLMs.
The secret sauce of ALFA? 🔍
6 key attributes from theory (cognitive science, medicine):
General:
- Clarity ✨
- Focus 🎯
- Answerability 💭
Clinical:
- Medical Accuracy 🏥
- Diagnostic Relevance 🔬
- Avoiding Bias ⚖️
Each attribute contributes to different aspect of the complex goal of question asking!
6 key attributes from theory (cognitive science, medicine):
General:
- Clarity ✨
- Focus 🎯
- Answerability 💭
Clinical:
- Medical Accuracy 🏥
- Diagnostic Relevance 🔬
- Avoiding Bias ⚖️
Each attribute contributes to different aspect of the complex goal of question asking!
February 21, 2025 at 4:03 PM
The secret sauce of ALFA? 🔍
6 key attributes from theory (cognitive science, medicine):
General:
- Clarity ✨
- Focus 🎯
- Answerability 💭
Clinical:
- Medical Accuracy 🏥
- Diagnostic Relevance 🔬
- Avoiding Bias ⚖️
Each attribute contributes to different aspect of the complex goal of question asking!
6 key attributes from theory (cognitive science, medicine):
General:
- Clarity ✨
- Focus 🎯
- Answerability 💭
Clinical:
- Medical Accuracy 🏥
- Diagnostic Relevance 🔬
- Avoiding Bias ⚖️
Each attribute contributes to different aspect of the complex goal of question asking!
📚 Exciting Dataset Release: MediQ-AskDocs!
17k real clinical interactions
80k attribute-specific question variations
302 expert-annotated scenarios
Perfect for research on interactive medical AI
First major dataset for training & evaluating medical question-asking! 🎯
huggingface.co/datasets/ste...
17k real clinical interactions
80k attribute-specific question variations
302 expert-annotated scenarios
Perfect for research on interactive medical AI
First major dataset for training & evaluating medical question-asking! 🎯
huggingface.co/datasets/ste...
stellalisy/MediQ_AskDocs_preference · Datasets at Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
February 21, 2025 at 4:02 PM
📚 Exciting Dataset Release: MediQ-AskDocs!
17k real clinical interactions
80k attribute-specific question variations
302 expert-annotated scenarios
Perfect for research on interactive medical AI
First major dataset for training & evaluating medical question-asking! 🎯
huggingface.co/datasets/ste...
17k real clinical interactions
80k attribute-specific question variations
302 expert-annotated scenarios
Perfect for research on interactive medical AI
First major dataset for training & evaluating medical question-asking! 🎯
huggingface.co/datasets/ste...
Introducing ALFA: ALignment via Fine-grained Attributes 🎓
A systematic, general question-asking framework that:
1️⃣ Decomposes the concept of good questioning into attributes📋
2️⃣ Generates targeted attribute-specific data📚
3️⃣ Teaches LLMs through preference learning🧑🏫
A systematic, general question-asking framework that:
1️⃣ Decomposes the concept of good questioning into attributes📋
2️⃣ Generates targeted attribute-specific data📚
3️⃣ Teaches LLMs through preference learning🧑🏫
February 21, 2025 at 4:01 PM
Introducing ALFA: ALignment via Fine-grained Attributes 🎓
A systematic, general question-asking framework that:
1️⃣ Decomposes the concept of good questioning into attributes📋
2️⃣ Generates targeted attribute-specific data📚
3️⃣ Teaches LLMs through preference learning🧑🏫
A systematic, general question-asking framework that:
1️⃣ Decomposes the concept of good questioning into attributes📋
2️⃣ Generates targeted attribute-specific data📚
3️⃣ Teaches LLMs through preference learning🧑🏫