This is joint work with Subhadip Mukherjee, Lindon Roberts, and Matthias J. Ehrhardt

#MachineLearning #Optimisation #Imaging #Bilevel_Learning

Our numerical experiments show:

📈 Significant speed-ups and better performance in image denoising and deblurring compared to the Method of Adaptive Inexact Descent (MAID).

Read the full preprint here:

arxiv.org/abs/2412.12049

Why does this matter?

🔍 Insights into the behaviour of inexact stochastic gradients in bilevel problems, with practical assumptions and convergence results.
✅ Faster performance vs. adaptive deterministic bilevel methods.
✅ Better generalisation for imaging tasks.

In this work, we make a theoretical contribution by connecting stochastic approximate hypergradients in bilevel optimisation to the theory of stochastic nonconvex optimisation.

Under mild assumptions, we prove these hypergradients satisfy the Biased ABC assumption for SGD.

This is a joint work with Lea Bogensperger, Matthias J. Ehrhardt, Thomas Pock, and Hok Shing Wong.

📊 Below, see how the learned ICNN regulariser performs on a sparse-angle computed tomography problem. Our bilevel framework shows significant improvement compared to adversarial training-based methods previously introduced in the literature.

✨ Key Highlights:
•A-posteriori error bounds for inexact hypergradients computed by primal-dual style differentiation.
•Adaptive, convergent bilevel framework with primal-dual style differentiation.
•Application to learning data-adaptive regularisers (e.g., ICNNs)!