Researcher in computational linguistics at University of Zurich
In multilingual models, the same meaning can take far more tokens in some languages, penalizing users of underrepresented languages with worse performance and higher API costs. Our Parity-aware BPE algorithm is a step toward addressing this issue: 🧵
In multilingual models, the same meaning can take far more tokens in some languages, penalizing users of underrepresented languages with worse performance and higher API costs. Our Parity-aware BPE algorithm is a step toward addressing this issue: 🧵
If you’re at #ACL, stop by to learn more!
If you’re at #ACL, stop by to learn more!
aclanthology.org/2025.acl-lon...
aclanthology.org/2025.acl-lon...