I'm not sure if I follow - your criticism is so broad that it is difficult to parse what specifically applies to conformal prediction vs. any statistical method that makes an assumption on the data-generating mechanism. Could you clarify?

So, to be precise, we can for example, consider a local reweighting around any point of interest to you, and provide a finite-sample coverage guarantee. Asking for true object-conditional coverage (without any assumptions) is impossible, but I think this is still a good middle ground.

Not to too shamelessly plug some work from your old group, but have you seen: arxiv.org/abs/2305.12616.

The guarantees can be substantially stronger than "on average over the future samples" now.