Our paper from the ICML 2025 workshop on Multi-Agent Systems explores a cascaded LLM framework for cost-effective decision-making. Small models defer to larger ones—or abstain for human input—based on uncertainty.

Paper: arxiv.org/abs/2506.11887
#LLM #MultiAgent #ICML2025

Our paper from the ICML 2025 workshop on Models of Human Feedback (Oral presentation) proposes few-shot steerable alignment for LLMs, utilising neural processes to handle conflicting and unobserved preferences.

Paper: arxiv.org/abs/2412.13998

#LLM #AIAlignment #ICML2025

To all NeurIPS warriors out there, good luck with the final sprint. Just one more day, we got this! :)

We teamed up with 🤗Hugging Face to release a free notebook for fine-tuning Gemma 3 with GRPO

Learn to:
• Enable reasoning in Gemma 3 (1B)
• Prepare/understand reward functions
• Make GRPO work for tiny LLMs

Notebook: colab.research.google.com/github/unslo...
Details: huggingface.co/reasoning-co...

Have a listen to Chris and Robert speak about our DiscoPOP paper (arxiv.org/abs/2406.08414) in the Machine Learning Street Talk (MLST) Podcast:

youtube.com/watch?v=1kwb...

dolphin-r1: a dataset for training R1-style models

- 800k total samples dataset similar in composition to the data used to train DeepSeek-R1 Distill models.
- 300k from DeepSeek-R1
- 300k from Gemini 2.0 flash thinking
- 200k from Dolphin chat

huggingface.co/datasets/cog...

Hot take: High-Flyer (the quant shop behind DeepSeek) orchestrated the release and hype of DeepSeek-R1 carefully, and made the big bugs today with shorts on NVDA🤑

huggingface is doing a fully open source replication of R1 github.com/huggingface/...

LM agents today primarily aim to automate tasks. Can we turn them into collaborative teammates? 🤖➕👤

Introducing Collaborative Gym (Co-Gym), a framework for enabling & evaluating human-agent collaboration! I now get used to agents proactively seeking confirmations or my deep thinking.(🧵 with video)

Qwen released a 72B process reward model (PRM) on their recent math model. A good chance it's the best PRM openly available for reasoning research. We like Qwen.
https://buff.ly/4gQV9wt

Yes!

Here are the slides for our language modeling tutorial with @kylelo.bsky.social and @akshitab.bsky.social in west ballroom b (ongoing).

docs.google.com/presentation...

I am at #NeurIPS this week, and presenting our poster "Discovering Preference Optimization Algorithms with and for Large Language Models" on Thursday at East Exhibit Hall A-C #3304.

Let me know if you want to chat about alignment, LLMs, and AI applications in medicine!

arxiv.org/abs/2406.08414

Ok, it is yesterdays news already, but good night sleep is important.

After 7 amazing years at Google Brain/DM, I am joining OpenAI. Together with @xzhai.bsky.social and @giffmana.ai, we will establish OpenAI Zurich office. Proud of our past work and looking forward to the future.

How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:

Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢

🧵⬇️

Excited to share our #NeurIPS 2024 Oral, Convolutional Differentiable Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA. Paper: arxiv.org/abs/2411.04732

🎉First #NeurIPS paper: We used LLMs to discover new preference optimization algorithms through evolution - they propose code, train a separate LLM & use the performance feedback to improve. After some generations, it found multiple novel, high-performing objective functions! arxiv.org/pdf/2406.08414

🚨 Adaptive Decoding via Latent Preference Optimization 🚨
- New layer for Transformer, selects decoding params automatically *per token*
- Learnt via new method Latent Preference Optimization
- Outperforms any fixed temperature decoding, choosing creativity or factuality
arxiv.org/abs/2411.09661
🧵1/4

LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams.

RLHF = 30% *more* copying than base!

Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩

arxiv.org/pdf/2410.04265