@howardhuo.bsky.social
Interesting to see evidence that LLMs aren't great at assessing difficulty. Classic example: Claude thinking every project takes 4-6 weeks.
LLMs Encode How Difficult Problems Are
LLMs Encode How Difficult Problems Are
Large language models exhibit a puzzling inconsistency: they solve complex problems yet frequently fail on seemingly simpler ones. We investigate whether LLMs internally encode problem difficulty in a way that aligns with human judgment, and whether this representation tracks generalization during r
arxiv.org
November 7, 2025 at 8:15 PM
Interesting to see evidence that LLMs aren't great at assessing difficulty. Classic example: Claude thinking every project takes 4-6 weeks.