What happens when an LLM is asked to use information that contradicts its knowledge? We explore knowledge conflict in a new preprint📑
TLDR: Performance drops, and this could affect the overall performance of LLMs in model-based evaluation.📑🧵⬇️ 1/8
#NLProc #LLM #AIResearch

Had so many fruitful discussions and made many friends this #NAACL2025 🌵🏜️Thanks for everyone who came to my poster or listened to me talking about my audacious thoughts! 😜

(I should have printed more stickers as they were more popular than I anticipated😅)

Dialects lie on continua of (structured) linguistic variation, right? And we can’t collect data for every point on the continuum...🤔
📢 Check out DialUp, a technique to make your MT model robust to the dialect continua of its training languages, including unseen dialects.
arxiv.org/abs/2501.16581

Meta literally created a LGBTQ exception for calling someone mentally ill as an insult. You can't do it for any other group except LGBTQ people.

with reasonable freedom, depending on the scale/focus of the business.

Case in point, we are looking to expand the research/foundation models team at Orby AI and are looking for highly motivated researchers and ML/Research engineers. Please reach out if you're interested in learning more!
/fin

Excited to start my #ARR #NLP reviews!

I'll try my best and see if I can get 100% of my reviews to be 'great' this round.

If you didn't see it already, ARR publishes how many of your reviews are considered to be 'great': stats.aclrollingreview.org

Join me for the challenge :)

🚨 I am on the faculty job market this year 🚨
I will be presenting at #NeurIPS2024 and am happy to chat in-person or digitally!

I work on developing AI agents that can collaborate and communicate robustly with us and each other.

More at: esteng.github.io and in thread below

🧵👇

Is MMLU Western-centric? 🤔

As part of a massive cross-institutional collaboration:
🗽Find MMLU is heavily overfit to western culture
🔍 Professional annotation of cultural sensitivity data
🌍 Release improved Global-MMLU 42 languages

📜 Paper: arxiv.org/pdf/2412.03304
📂 Data: hf.co/datasets/Coh...

🚨I too am on the job market‼️🤯

I'm searching for faculty positions/postdocs in multilingual/multicultural NLP, vision+language models, and eval for genAI!

I'll be at #NeurIPS2024 presenting our work on meta-evaluation for text-to-image faithfulness! Let's chat there!

Papers in🧵, see more: saxon.me

Excited to share OLMo 2!

🐟 7B and 13B weights, trained up to 4-5T tokens, fully open data, code, etc
🐠 better architecture and recipe for training stability
🐡 staged training, with new data mix Dolmino🍕 added during annealing
🦈 state-of-the-art OLMo 2 Instruct models

#nlp #mlsky

links below👇

Putting together a JHU Center for Language and Speech Processing starter pack!

Please reply or DM me if you're doing research at CLSP and would like to be added - I'm still trying to find out which of us are on here so far.

go.bsky.app/JtWKca2

A starter pack for #NLP #NLProc researchers! 🎉

go.bsky.app/SngwGeS

Dealing with a new social media account can be vexatious. Here I compiled a thread of resources that might be helpful to transition to Bluesky 🦋 . 🧵⬇️ Thread below