Rolf Pendall
@rpplan.bsky.social
I'm an urban planning educator at the University of Illinois, Urbana-Champaign. My questions/quests: Can zoning make communities equitable instead of protecting privilege? What narratives & political campaigns can lead to zoning for equity?
LLMs suck at lit review, and they're getting worse at it. Transparently obvious from today's news (HHS's nonsense MAHA report!) and backed with recently published peer-reviewed research by Peters & Chin-Yee: doi.org/10.1098/rsos...
![Abstract: "Artificial intelligence chatbots driven by large language
models (LLMs) have the potential to increase public science
literacy and support scientific research, as they can quickly
summarize complex scientific information in accessible terms.
However, when summarizing scientific texts, LLMs may
omit details that limit the scope of research conclusions,
leading to generalizations of results broader than warranted
by the original study. We tested 10 prominent LLMs,
including ChatGPT-4o, ChatGPT-4.5, DeepSeek, LLaMA 3.3
70B, and Claude 3.7 Sonnet, comparing 4900 LLM-generated
summaries to their original scientific texts. Even when
explicitly prompted for accuracy, most LLMs produced
broader generalizations of scientific results than those in
the original texts, with DeepSeek, ChatGPT-4o, and LLaMA
3.3 70B overgeneralizing in 26–73% of cases. In a direct
comparison of LLM-generated and human-authored science
summaries, LLM summaries were nearly five times more
likely to contain broad generalizations (odds ratio = 4.85,
95% CI [3.06, 7.70], p < 0.001). Notably, newer models tended
to perform worse in generalization accuracy than earlier
ones. Our results indicate a strong bias in many widely
used LLMs towards overgeneralizing scientific conclusions,
posing a significant risk of large-scale misinterpretations
of research findings. We highlight potential mitigation
strategies, including lowering LLM temperature settings and
benchmarking LLMs for generalization accuracy."](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:qiahueu7wxuzarfakwmp2cjz/bafkreihfxekxi2bwo6gq7hoqmv2isumlghcg2nnysb6ssmhvx75b2onhum@jpeg)
May 30, 2025 at 7:15 PM
LLMs suck at lit review, and they're getting worse at it. Transparently obvious from today's news (HHS's nonsense MAHA report!) and backed with recently published peer-reviewed research by Peters & Chin-Yee: doi.org/10.1098/rsos...
Nice that the Champaign Park District has the equipment to plow the path at Hessel Park but it’s only safe to get there in a car. OTOH at least the streets are a mile wide and plowed curb to curb so peds can say to heck with the sidewalk
January 19, 2025 at 12:44 AM
Nice that the Champaign Park District has the equipment to plow the path at Hessel Park but it’s only safe to get there in a car. OTOH at least the streets are a mile wide and plowed curb to curb so peds can say to heck with the sidewalk
What drivers get | What walkers get, Champaign “in-town” edition, I.e., the area that’s supposed to be pedestrian-friendly
January 19, 2025 at 12:38 AM
What drivers get | What walkers get, Champaign “in-town” edition, I.e., the area that’s supposed to be pedestrian-friendly