How can we use models of cognition to help LLMs interpret figurative language (irony, hyperbole) in a more human-like manner? Come to our #ACL2025NLP poster on Wednesday at 11AM (exhibit hall - exact location TBA) to find out! @mcgill-nlp.bsky.social @mila-quebec.bsky.social @aclmeeting.bsky.social

What about LLMs? We integrate LLMs within (RSA)^2 and test them on a new dataset, PragMega+. We show that LLMs augmented with (RSA)^2 produce probability distributions which are more aligned with human expectations.

We test (RSA)^2 on two existing figurative language datasets: hyperbolic number expressions (e.g. “This kettle costs 1000$”) and ironic utterances about the weather (e.g. “The weather is amazing” during a Montreal blizzard). We obtain meaning distributions which are compatible with those of humans!

We develop (RSA)^2: a 𝘳𝘩𝘦𝘵𝘰𝘳𝘪𝘤𝘢𝘭-𝘴𝘵𝘳𝘢𝘵𝘦𝘨𝘺-𝘢𝘸𝘢𝘳𝘦 probabilistic framework of figurative language. In (RSA)^2 one listener will interpret language literally, another will interpret language ironically, etc. These listeners are marginalized to produce a distribution over possible meanings.

A blizzard is raging through Montreal when your friend says “Looks like Florida out there!” Humans easily interpret irony, while LLMs struggle with it. We propose a 𝘳𝘩𝘦𝘵𝘰𝘳𝘪𝘤𝘢𝘭-𝘴𝘵𝘳𝘢𝘵𝘦𝘨𝘺-𝘢𝘸𝘢𝘳𝘦 probabilistic framework as a solution.
Paper: arxiv.org/abs/2506.09301 to appear @ #ACL2025 (Main)