Lukas Edman
@lukasnlp.bsky.social
Post-Doc at LMU Munich, interested in just about anything in low-resource NLP
Happy to announce two papers at #ACL2025!
1. We made an extension to our CUTE benchmark, EXECUTE! We find that LLMs actually *do* have the ability to do character-level manipulation, but language bias, and tokenization too, get in the way. openreview.net/forum?id=m95...
1/2
1. We made an extension to our CUTE benchmark, EXECUTE! We find that LLMs actually *do* have the ability to do character-level manipulation, but language bias, and tokenization too, get in the way. openreview.net/forum?id=m95...
1/2
EXECUTE: A Multilingual Benchmark for LLM Token Understanding
The CUTE benchmark showed that LLMs struggle with character understanding in English. We extend it to more languages with diverse scripts and writing systems, introducing EXECUTE. Our simplified...
openreview.net
May 16, 2025 at 9:49 PM
Happy to announce two papers at #ACL2025!
1. We made an extension to our CUTE benchmark, EXECUTE! We find that LLMs actually *do* have the ability to do character-level manipulation, but language bias, and tokenization too, get in the way. openreview.net/forum?id=m95...
1/2
1. We made an extension to our CUTE benchmark, EXECUTE! We find that LLMs actually *do* have the ability to do character-level manipulation, but language bias, and tokenization too, get in the way. openreview.net/forum?id=m95...
1/2