Tom Kocmi
@kocmitom.bsky.social
Researcher at Cohere | Multilingual LLM evaluation
🚀 Thrilled to share what I’ve been working on at Cohere!
What began in January as a scribble in my notebook “how challenging would it be...” turned into a fully-fledged translation model that outperforms both open and closed-source systems, including long-standing MT leaders.
What began in January as a scribble in my notebook “how challenging would it be...” turned into a fully-fledged translation model that outperforms both open and closed-source systems, including long-standing MT leaders.
August 28, 2025 at 7:55 PM
🚀 Thrilled to share what I’ve been working on at Cohere!
What began in January as a scribble in my notebook “how challenging would it be...” turned into a fully-fledged translation model that outperforms both open and closed-source systems, including long-standing MT leaders.
What began in January as a scribble in my notebook “how challenging would it be...” turned into a fully-fledged translation model that outperforms both open and closed-source systems, including long-standing MT leaders.
🏆 Highlights from top systems:
✅ IOL-Research: led in constrained/open, winning 10/11 in its category.
✅ Unbabel-Tower70B: Best participant, winning 8/11 pairs.
✅ Claude-3.5-Sonnet: Best overall with 9/11 wins.
✅ Shoutout to Dubformer (speech) & CUNI-MH (strong constrained)
✅ IOL-Research: led in constrained/open, winning 10/11 in its category.
✅ Unbabel-Tower70B: Best participant, winning 8/11 pairs.
✅ Claude-3.5-Sonnet: Best overall with 9/11 wins.
✅ Shoutout to Dubformer (speech) & CUNI-MH (strong constrained)
November 20, 2024 at 10:16 AM
🏆 Highlights from top systems:
✅ IOL-Research: led in constrained/open, winning 10/11 in its category.
✅ Unbabel-Tower70B: Best participant, winning 8/11 pairs.
✅ Claude-3.5-Sonnet: Best overall with 9/11 wins.
✅ Shoutout to Dubformer (speech) & CUNI-MH (strong constrained)
✅ IOL-Research: led in constrained/open, winning 10/11 in its category.
✅ Unbabel-Tower70B: Best participant, winning 8/11 pairs.
✅ Claude-3.5-Sonnet: Best overall with 9/11 wins.
✅ Shoutout to Dubformer (speech) & CUNI-MH (strong constrained)
📊 We introduced new robust and efficient human evaluation protocol: Error Span Annotations (ESA).
📄 Test sets are now finally document-level!
🌍 We've added three new language pairs, including English-Spanish where translations are near-perfect.
For more details, read our findings paper.
📄 Test sets are now finally document-level!
🌍 We've added three new language pairs, including English-Spanish where translations are near-perfect.
For more details, read our findings paper.
November 20, 2024 at 10:16 AM
📊 We introduced new robust and efficient human evaluation protocol: Error Span Annotations (ESA).
📄 Test sets are now finally document-level!
🌍 We've added three new language pairs, including English-Spanish where translations are near-perfect.
For more details, read our findings paper.
📄 Test sets are now finally document-level!
🌍 We've added three new language pairs, including English-Spanish where translations are near-perfect.
For more details, read our findings paper.
Exciting time at this year's WMT24 General MT Shared Task:
🚀 Participant numbers increased by over 50%!
🏗️ Decoder-only architectures are leading the way.
🔊 We've introduced a new speech audio modality domain.
🌐 Online systems are losing ground to LLMs.
🚀 Participant numbers increased by over 50%!
🏗️ Decoder-only architectures are leading the way.
🔊 We've introduced a new speech audio modality domain.
🌐 Online systems are losing ground to LLMs.
November 20, 2024 at 10:16 AM
Exciting time at this year's WMT24 General MT Shared Task:
🚀 Participant numbers increased by over 50%!
🏗️ Decoder-only architectures are leading the way.
🔊 We've introduced a new speech audio modality domain.
🌐 Online systems are losing ground to LLMs.
🚀 Participant numbers increased by over 50%!
🏗️ Decoder-only architectures are leading the way.
🔊 We've introduced a new speech audio modality domain.
🌐 Online systems are losing ground to LLMs.