In this paper we propose strategic conformal prediction, a simple extension of the standard conformal prediction methods which ensures that the uncertainty quantification is robust to strategic alterations.

Happy to announce that our paper, 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗰 𝗖𝗼𝗻𝗳𝗼𝗿𝗺𝗮𝗹 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻, is now accepted to AISTATS 2025!

Is your uncertainty quantification robust to people trying to break it?
Ours is :)

Anthropic published a paper telling people to use confidence intervals for evals.

I now await for their next paper, which will explain multiple comparisons to the LLM people

Gradient descent with random initialization avoids troublesome stationary points almost surely.

#🧮

Useful fact: you can bound the gap between any two probability distributions using f-divergences. A rather nice case is with the Pearson chi-squared f-divergence.