Implications vary by task and domain. Researchers should clearly define their annotation constructs before reviewing LLM annotations. We are subject to anchoring bias that can affect our evaluations, or even our research findings!
Read more: arxiv.org/abs/2507.15821

What happens if we use LLM-influenced labels as ground truth when evaluating LLM performance on these tasks? We can seriously overestimate LLM performance on these tasks. F1 scores for some tasks were +.5 higher when evaluated using LLM-influenced labels as ground truth!

However… annotators STRONGLY took LLM suggestions: just 40% of human crowd labels overlap with LLM baselines, but overlap jumps to over 80% when LLM suggestions are given (varied crowd thresholds and conditions shown in graph). Beware: Humans are subject to anchoring bias!

*Are disclosed features connected to described impacts?* Disclosed features are much less commonly described in terms of impacts the feature had on the research, which may leave a gap for readers to jump to conclusions about how the disclosed feature impacted the work.

We reviewed papers from the entire history of FAccT for the presence of positionality statements. We find 2024 marked a significant proportional increase in papers that included positionality statements, likely as a result of PC recommendations:

Great morning presenting LBW at #CHI2025: Forage: Understanding RAG-based Sensemaking for Community Conversations!

Assistance in generating a pesky table is different than generating a whole abstract... right? Curious if anyone has used this and what you think!

Entered Overleaf today- "Writefull" is directly integrated and can edit and even write my draft with AI.
Presented as something to opt out of, ethical, transparent engagement with these tools is increasingly difficult, especially as venue/institutional norms on disclosure change by the day.