Reza Madani
@rezamadani.bsky.social
Researcher @UniTrento | Former MSc Student @UniBo | Former Visiting PGR Student @EdinburghUni | Natural Language Processing, Large Language Models. https://qasemii.github.io/
Reposted by Reza Madani
MMLU-Redux just touched down at #NAACL2025! 🎉
Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅
If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋
Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅
If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋
May 2, 2025 at 1:00 PM
MMLU-Redux just touched down at #NAACL2025! 🎉
Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅
If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋
Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅
If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋
"Are We Done with MMLU" made it to #NAACL2025. Massive congrats to the team especially @aryopg.bsky.social. 🚀
📋 Preprint: arxiv.org/abs/2406.04127
👨🏻💻 GitHub: github.com/aryopg/mmlu-...
🤗 HuggingFace: huggingface.co/datasets/edi...
📋 Preprint: arxiv.org/abs/2406.04127
👨🏻💻 GitHub: github.com/aryopg/mmlu-...
🤗 HuggingFace: huggingface.co/datasets/edi...
Are We Done with MMLU?
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates numerous ground truth ...
arxiv.org
January 23, 2025 at 12:18 PM
"Are We Done with MMLU" made it to #NAACL2025. Massive congrats to the team especially @aryopg.bsky.social. 🚀
📋 Preprint: arxiv.org/abs/2406.04127
👨🏻💻 GitHub: github.com/aryopg/mmlu-...
🤗 HuggingFace: huggingface.co/datasets/edi...
📋 Preprint: arxiv.org/abs/2406.04127
👨🏻💻 GitHub: github.com/aryopg/mmlu-...
🤗 HuggingFace: huggingface.co/datasets/edi...
Reposted by Reza Madani
For clarity -- great project, but most of the MMLU errors we found (and fixed) in our MMLU Redux paper (arxiv.org/abs/2406.04127) are also present in this dataset. We also provide a curated version of MMLU, so it's easy to fix 😊
Announcing Global-MMLU - an improved MMLU Open dataset with evaluation coverage across 42 languages.
The result of months of work with the goal of advancing Multilingual LLM evaluation.
Built together with the community and amazing collaborators at Cohere4AI, MILA, MIT, and many more.
The result of months of work with the goal of advancing Multilingual LLM evaluation.
Built together with the community and amazing collaborators at Cohere4AI, MILA, MIT, and many more.
December 6, 2024 at 9:26 AM
For clarity -- great project, but most of the MMLU errors we found (and fixed) in our MMLU Redux paper (arxiv.org/abs/2406.04127) are also present in this dataset. We also provide a curated version of MMLU, so it's easy to fix 😊