what😳

Huge thanks to my amazing co-author @marcusklasson.bsky.social @arnosolin.bsky.social @trappmartin.bsky.social 🙌

Also: I’m looking for research internship! If you're working on uncertainty estimation, Bayesian methods, or large models, I’d love to connect.

Our method scales to 𝐕𝐢𝐓, performs well, and provides useful predictive uncertainties. You can try it out with our open-source library SUQ (github.com/AaltoML/SUQ).

So we ask:
👉 𝐖𝐡𝐚𝐭 𝐢𝐟 𝐩𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 𝐜𝐨𝐮𝐥𝐝 𝐛𝐞 𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐚𝐧𝐝 𝐞𝐟𝐟𝐞𝐜𝐭𝐢𝐯𝐞 — 𝐰𝐢𝐭𝐡𝐨𝐮𝐭 𝐬𝐚𝐦𝐩𝐥𝐢𝐧𝐠?
We show that with
• Local linearisation of activation functions
• Local Gaussian approximations in linear layers
We can 𝐚𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐚𝐥𝐥𝐲 compute an approximation to the posterior predictive with a 𝐬𝐢𝐧𝐠𝐥𝐞 𝐟𝐨𝐫𝐰𝐚𝐫𝐝 𝐩𝐚𝐬𝐬.

In Bayesian deep learning, we spend a lot of effort getting the 𝘱𝘰𝘴𝘵𝘦𝘳𝘪𝘰𝘳 right — setting correct priors, better approximate inference — you name it. But when it comes to 𝘱𝘳𝘦𝘥𝘪𝘤𝘵𝘪𝘰𝘯, we still mostly rely on Monte Carlo estimation. It's slow, noisy, and often throws away what the posterior learned.