Michael Kirchhof (ICML)
@mkirchhof.bsky.social
Research Scientist at Apple for uncertainty quantification.
Pinned
Can LLMs access and describe their own internal distributions? With my colleagues at Apple, I invite you to take a leap forward and make LLM uncertainty quantification what it can be.
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
Our research team is hiring PhD interns 🍏 Spend your next summer in Paris and explore the next frontiers of LLMs for uncertainty quantification, calibration, RL and post-training, and Bayesian experimental design.
Details & Application ➡️ jobs.apple.com/en-my/detail...
Details & Application ➡️ jobs.apple.com/en-my/detail...
Internship - Machine Learning Research on Uncertainty - Jobs at Apple (MY)
Apply for a Internship - Machine Learning Research on Uncertainty job at Apple. Read about the role and find out if it’s right for you.
jobs.apple.com
November 14, 2025 at 4:26 PM
Our research team is hiring PhD interns 🍏 Spend your next summer in Paris and explore the next frontiers of LLMs for uncertainty quantification, calibration, RL and post-training, and Bayesian experimental design.
Details & Application ➡️ jobs.apple.com/en-my/detail...
Details & Application ➡️ jobs.apple.com/en-my/detail...
Reposted by Michael Kirchhof (ICML)
📢 We’re looking for a researcher in in cogsci, neuroscience, linguistics, or related disciplines to work with us at Apple Machine Learning Research! We're hiring for a one-year interdisciplinary AIML Resident to work on understanding reasoning and decision making in LLMs. 🧵
November 7, 2025 at 9:19 PM
📢 We’re looking for a researcher in in cogsci, neuroscience, linguistics, or related disciplines to work with us at Apple Machine Learning Research! We're hiring for a one-year interdisciplinary AIML Resident to work on understanding reasoning and decision making in LLMs. 🧵
Reposted by Michael Kirchhof (ICML)
We have been working with Michal Klein on pushing a module to train *flow matching* models using JAX. This is shipped as part of our new release of the OTT-JAX toolbox (github.com/ott-jax/ott)
The tutorial to do so is here: ott-jax.readthedocs.io/tutorials/ne...
The tutorial to do so is here: ott-jax.readthedocs.io/tutorials/ne...
November 5, 2025 at 2:04 PM
We have been working with Michal Klein on pushing a module to train *flow matching* models using JAX. This is shipped as part of our new release of the OTT-JAX toolbox (github.com/ott-jax/ott)
The tutorial to do so is here: ott-jax.readthedocs.io/tutorials/ne...
The tutorial to do so is here: ott-jax.readthedocs.io/tutorials/ne...
Reposted by Michael Kirchhof (ICML)
It's that time of the year! 🎁
The Apple Machine Learning Research (MLR) team in Paris is hiring a few interns, to do cool research for ±6 months 🚀🚀 & work towards publications/OSS.
Check requirements and apply: ➡️ jobs.apple.com/en-us/detail...
More❓→ ✉️ mlr_paris_internships@group.apple.com
The Apple Machine Learning Research (MLR) team in Paris is hiring a few interns, to do cool research for ±6 months 🚀🚀 & work towards publications/OSS.
Check requirements and apply: ➡️ jobs.apple.com/en-us/detail...
More❓→ ✉️ mlr_paris_internships@group.apple.com
October 17, 2025 at 1:07 PM
It's that time of the year! 🎁
The Apple Machine Learning Research (MLR) team in Paris is hiring a few interns, to do cool research for ±6 months 🚀🚀 & work towards publications/OSS.
Check requirements and apply: ➡️ jobs.apple.com/en-us/detail...
More❓→ ✉️ mlr_paris_internships@group.apple.com
The Apple Machine Learning Research (MLR) team in Paris is hiring a few interns, to do cool research for ±6 months 🚀🚀 & work towards publications/OSS.
Check requirements and apply: ➡️ jobs.apple.com/en-us/detail...
More❓→ ✉️ mlr_paris_internships@group.apple.com
LLMs are currently this one big parameter block that stores all sort of facts. In our new preprint, we add context-specific memory parameters to the model, and pretrain the model along with a big bank of memories.
📑 arxiv.org/abs/2510.02375
[1/10]🧵
📑 arxiv.org/abs/2510.02375
[1/10]🧵
October 6, 2025 at 4:06 PM
LLMs are currently this one big parameter block that stores all sort of facts. In our new preprint, we add context-specific memory parameters to the model, and pretrain the model along with a big bank of memories.
📑 arxiv.org/abs/2510.02375
[1/10]🧵
📑 arxiv.org/abs/2510.02375
[1/10]🧵
Reposted by Michael Kirchhof (ICML)
Our two phenomenal interns, Alireza Mousavi-Hosseini and Stephen Zhang @syz.bsky.social have been cooking some really cool work with Michal Klein and me over the summer.
Relying on optimal transport couplings (to pick noise and data pairs) should, in principle, be helpful to guide flow matching
🧵
Relying on optimal transport couplings (to pick noise and data pairs) should, in principle, be helpful to guide flow matching
🧵
October 3, 2025 at 8:50 PM
Our two phenomenal interns, Alireza Mousavi-Hosseini and Stephen Zhang @syz.bsky.social have been cooking some really cool work with Michal Klein and me over the summer.
Relying on optimal transport couplings (to pick noise and data pairs) should, in principle, be helpful to guide flow matching
🧵
Relying on optimal transport couplings (to pick noise and data pairs) should, in principle, be helpful to guide flow matching
🧵
Many treat uncertainty = a number. At Apple, we're rethinking this: LLMs should output strings that reveal all information of their internal distributions. We find that Reasoning, SFT, CoT can't do it - yet. To get there, we introduce the SelfReflect benchmark.
arxiv.org/pdf/2505.20295
arxiv.org/pdf/2505.20295
October 1, 2025 at 9:53 AM
Many treat uncertainty = a number. At Apple, we're rethinking this: LLMs should output strings that reveal all information of their internal distributions. We find that Reasoning, SFT, CoT can't do it - yet. To get there, we introduce the SelfReflect benchmark.
arxiv.org/pdf/2505.20295
arxiv.org/pdf/2505.20295
Reposted by Michael Kirchhof (ICML)
Natural idea. Looks like a nice paper too. arxiv.org/abs/2508.21184
BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design
We propose a general-purpose approach for improving the ability of Large Language Models (LLMs) to intelligently and adaptively gather information from a user or other external source using the framew...
arxiv.org
September 1, 2025 at 12:17 PM
Natural idea. Looks like a nice paper too. arxiv.org/abs/2508.21184
I'll present my view on the future of uncertainties in LLMs and vision models at @icmlconf.bsky.social, in penal discussions, posters, and workshops. Reach out if you wanna chat :)
Here's everything from me and other folks at Apple: machinelearning.apple.com/updates/appl...
Here's everything from me and other folks at Apple: machinelearning.apple.com/updates/appl...
July 13, 2025 at 11:47 PM
I'll present my view on the future of uncertainties in LLMs and vision models at @icmlconf.bsky.social, in penal discussions, posters, and workshops. Reach out if you wanna chat :)
Here's everything from me and other folks at Apple: machinelearning.apple.com/updates/appl...
Here's everything from me and other folks at Apple: machinelearning.apple.com/updates/appl...
Can LLMs access and describe their own internal distributions? With my colleagues at Apple, I invite you to take a leap forward and make LLM uncertainty quantification what it can be.
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
July 3, 2025 at 9:08 AM
Can LLMs access and describe their own internal distributions? With my colleagues at Apple, I invite you to take a leap forward and make LLM uncertainty quantification what it can be.
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
📄 arxiv.org/abs/2505.20295
💻 github.com/apple/ml-sel...
🧵1/9
Reposted by Michael Kirchhof (ICML)
NEW PAPER ALERT: Recent studies have shown that LLMs often lack robustness to distribution shifts in their reasoning. Our paper proposes a new method, AbstRaL, to augment LLMs’ reasoning robustness, by promoting their abstract thinking with granular reinforcement learning.
June 23, 2025 at 2:32 PM
NEW PAPER ALERT: Recent studies have shown that LLMs often lack robustness to distribution shifts in their reasoning. Our paper proposes a new method, AbstRaL, to augment LLMs’ reasoning robustness, by promoting their abstract thinking with granular reinforcement learning.
I‘ll talk today about our latest research on uncertainty quantification at Apple (papers are 2 weeks old) and what I see as the future for UQ in vision and LLMs. See you at 102B, 4:30pm! PS: Lmk if you wanna chat :)
June 11, 2025 at 3:00 PM
I‘ll talk today about our latest research on uncertainty quantification at Apple (papers are 2 weeks old) and what I see as the future for UQ in vision and LLMs. See you at 102B, 4:30pm! PS: Lmk if you wanna chat :)
At the end of my PhD, I reflected on uncertainty quantification research, and what might change with chatbots and LLM agents. This was now accepted as position paper at @icmlconf.bsky.social. Some of those future topics are already picking up pace, so have an evening read ☕ arxiv.org/abs/2505.22655
May 29, 2025 at 7:30 AM
At the end of my PhD, I reflected on uncertainty quantification research, and what might change with chatbots and LLM agents. This was now accepted as position paper at @icmlconf.bsky.social. Some of those future topics are already picking up pace, so have an evening read ☕ arxiv.org/abs/2505.22655
Reposted by Michael Kirchhof (ICML)
Now that @interspeech.bsky.social registration is open, time for some shameless promo!
Sign-up and join our Interspeech tutorial: Speech Technology Meets Early Language Acquisition: How Interdisciplinary Efforts Benefit Both Fields. 🗣️👶
www.interspeech2025.org/tutorials
⬇️ (1/2)
Sign-up and join our Interspeech tutorial: Speech Technology Meets Early Language Acquisition: How Interdisciplinary Efforts Benefit Both Fields. 🗣️👶
www.interspeech2025.org/tutorials
⬇️ (1/2)
https://www.interspeech2025.org/tutorials
Your cookies are disabled, please enable them.
www.interspeech2025.org
May 27, 2025 at 4:14 PM
Now that @interspeech.bsky.social registration is open, time for some shameless promo!
Sign-up and join our Interspeech tutorial: Speech Technology Meets Early Language Acquisition: How Interdisciplinary Efforts Benefit Both Fields. 🗣️👶
www.interspeech2025.org/tutorials
⬇️ (1/2)
Sign-up and join our Interspeech tutorial: Speech Technology Meets Early Language Acquisition: How Interdisciplinary Efforts Benefit Both Fields. 🗣️👶
www.interspeech2025.org/tutorials
⬇️ (1/2)
Aleatoric and epistemic uncertainty are clear-cut concepts, right? ... right? 😵💫 In our new ICLR blogpost we let different schools of thought speak and contradict each other, and revisit chatbots where “the character of aleatory ‘transforms’ into epistemic” iclr-blogposts.github.io/2025/blog/re...
May 8, 2025 at 8:18 AM
Aleatoric and epistemic uncertainty are clear-cut concepts, right? ... right? 😵💫 In our new ICLR blogpost we let different schools of thought speak and contradict each other, and revisit chatbots where “the character of aleatory ‘transforms’ into epistemic” iclr-blogposts.github.io/2025/blog/re...
Reposted by Michael Kirchhof (ICML)
Today we have released the code and a demo iOS application for FastVLM - our extremely efficient and fast vision language model which runs on your device using MLX! You can check out the code and the app here: github.com/apple/ml-fas...
May 7, 2025 at 10:20 PM
Today we have released the code and a demo iOS application for FastVLM - our extremely efficient and fast vision language model which runs on your device using MLX! You can check out the code and the app here: github.com/apple/ml-fas...
Reposted by Michael Kirchhof (ICML)
Paper🧵 (cross-posted at X): When does composition of diffusion models "work"? Intuitively, the reason dog+hat works and dog+horse doesn’t has something to do with independence between the concepts being composed. The tricky part is to formalize exactly what this means. 1/
February 11, 2025 at 5:59 AM
Paper🧵 (cross-posted at X): When does composition of diffusion models "work"? Intuitively, the reason dog+hat works and dog+horse doesn’t has something to do with independence between the concepts being composed. The tricky part is to formalize exactly what this means. 1/
Reposted by Michael Kirchhof (ICML)
🚨 Apple Machine Learning Research Internship opportunity! My colleagues in Apple MLR are looking for a PhD research intern with a strong interest in reinforcement learning/post-training for LLMs. If interested, apply by sending an email to Etai Littwin (elittwin at apple dot com)
February 7, 2025 at 11:41 PM
🚨 Apple Machine Learning Research Internship opportunity! My colleagues in Apple MLR are looking for a PhD research intern with a strong interest in reinforcement learning/post-training for LLMs. If interested, apply by sending an email to Etai Littwin (elittwin at apple dot com)
Wow, OpenAI's o1 has a whopping 93% ECE on Humanity's Last Exam. So if you just prompt o1 to tell you how sure it is about its answer, it will basically produce gibberish. And that's how most users will ask for uncertainties. We have work to do!
January 24, 2025 at 9:41 AM
Wow, OpenAI's o1 has a whopping 93% ECE on Humanity's Last Exam. So if you just prompt o1 to tell you how sure it is about its answer, it will basically produce gibberish. And that's how most users will ask for uncertainties. We have work to do!
Reposted by Michael Kirchhof (ICML)
Today is a great day for optimal transport 🎉! Lots of gratitude 🙏 for all folks who contributed to ott-jax.readthedocs.io and pushed for the MOSCOT (now @ nature!) paper, from visionaries @dominik1klein.bsky.social, G. Palla, Z. Piran to the magician, Michal Klein! ❤️
www.nature.com/articles/s41...
www.nature.com/articles/s41...
January 22, 2025 at 10:18 PM
Today is a great day for optimal transport 🎉! Lots of gratitude 🙏 for all folks who contributed to ott-jax.readthedocs.io and pushed for the MOSCOT (now @ nature!) paper, from visionaries @dominik1klein.bsky.social, G. Palla, Z. Piran to the magician, Michal Klein! ❤️
www.nature.com/articles/s41...
www.nature.com/articles/s41...
Many LLM uncertainty estimators perform similarly, but does that mean they do the same? No! We find that they use different cues, and combining them gives even better performance. 🧵1/5
📄 openreview.net/forum?id=QKR...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
📄 openreview.net/forum?id=QKR...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
December 13, 2024 at 12:37 PM
Many LLM uncertainty estimators perform similarly, but does that mean they do the same? No! We find that they use different cues, and combining them gives even better performance. 🧵1/5
📄 openreview.net/forum?id=QKR...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
📄 openreview.net/forum?id=QKR...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
Reposted by Michael Kirchhof (ICML)
Interested in learning how to evaluate uncertainty in LLMs?
Check out our work at NeurIPS!
Feel free to reach out for a chat!
Check out our work at NeurIPS!
Feel free to reach out for a chat!
Evaluating your LLM uncertainties with Rougle-L will show clear winners... except that they aren't actually good. We find that Rouge-L spuriously favors some methods over others. 🧵1/4
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
December 12, 2024 at 5:12 PM
Interested in learning how to evaluate uncertainty in LLMs?
Check out our work at NeurIPS!
Feel free to reach out for a chat!
Check out our work at NeurIPS!
Feel free to reach out for a chat!
Reposted by Michael Kirchhof (ICML)
Excited to present our spotlight paper on uncertainty disentanglement at #NeurIPS! Drop by today between 11 am and 2 pm PST at West Ballroom A-D #5509 and let's chat!
December 12, 2024 at 6:00 PM
Excited to present our spotlight paper on uncertainty disentanglement at #NeurIPS! Drop by today between 11 am and 2 pm PST at West Ballroom A-D #5509 and let's chat!
Evaluating your LLM uncertainties with Rougle-L will show clear winners... except that they aren't actually good. We find that Rouge-L spuriously favors some methods over others. 🧵1/4
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
December 12, 2024 at 11:36 AM
Evaluating your LLM uncertainties with Rougle-L will show clear winners... except that they aren't actually good. We find that Rouge-L spuriously favors some methods over others. 🧵1/4
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
📄 openreview.net/forum?id=jGt...
NeurIPS: Sunday, East Exhibition Hall A, Safe Gen AI workshop
Reposted by Michael Kirchhof (ICML)
Ok, it is yesterdays news already, but good night sleep is important.
After 7 amazing years at Google Brain/DM, I am joining OpenAI. Together with @xzhai.bsky.social and @giffmana.ai, we will establish OpenAI Zurich office. Proud of our past work and looking forward to the future.
After 7 amazing years at Google Brain/DM, I am joining OpenAI. Together with @xzhai.bsky.social and @giffmana.ai, we will establish OpenAI Zurich office. Proud of our past work and looking forward to the future.
December 4, 2024 at 9:14 AM
Ok, it is yesterdays news already, but good night sleep is important.
After 7 amazing years at Google Brain/DM, I am joining OpenAI. Together with @xzhai.bsky.social and @giffmana.ai, we will establish OpenAI Zurich office. Proud of our past work and looking forward to the future.
After 7 amazing years at Google Brain/DM, I am joining OpenAI. Together with @xzhai.bsky.social and @giffmana.ai, we will establish OpenAI Zurich office. Proud of our past work and looking forward to the future.