(6/n) 🛠️ We propose DPAF — a Dual-Process Audit Framework — to probe both LLM decisions and justifications, ensuring fairness & transparency in high-stakes AI.

(5/n) 🤔 Asking LLMs to justify decisions flips outcomes: they often “rescue” low-performing, low-SES students while penalizing high-SES ones. Reasoning matters.

(4/n) 📊 LLMs consistently favor low-SES applicants—even controlling for performance. But sometimes penalize strong applicants without hardship markers.

(3/n) 🚨 We audited 4 leading LLMs this dataset. Result: models show clear patterns in how they weigh socioeconomic status (SES).

(2/n) 📚 We built a rigorous synthetic dataset of 30K applicant profiles, grounded in real-world correlations (SAT, GPA, SES). This let us audit LLMs in a controlled, large-scale way without risking student privacy

HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via LLMs

There are lots of hate datasets with different nuances. We show how to pretrain on a COT-enhanced dataset to get great performance on data du jour.

by Huy Nghiem +al EMNLP’24
hal3.name/docs/daume24...
>