Chitta Baral
@cbaral.bsky.social
Professor in Computer Science, Arizona State University
@garymarcus.bsky.social LLMs should be able to do correct logical reasoning of a decent sequence of reasoning steps. Our ACL 2024 paper arxiv.org/abs/2404.15522 evaluates their ability to do just one-step reasoning corresponding to inference rules in propositional logic, first-order logic, and more.
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks. But, can they really "reason" over the natural language? Thi...
arxiv.org
November 19, 2024 at 7:49 PM
@garymarcus.bsky.social LLMs should be able to do correct logical reasoning of a decent sequence of reasoning steps. Our ACL 2024 paper arxiv.org/abs/2404.15522 evaluates their ability to do just one-step reasoning corresponding to inference rules in propositional logic, first-order logic, and more.