Isn't there a lot of noise in these decisions, just like for conference papers etc?

Grattis!

The waiting area is also quite dull and gets really crowded, probably the worst part of my entire trip.

I think it was already in the batch of papers I was given to rate, basically no pathology-related papers, for example. ICML was definitely better in this regard.

Looks very useful, thanks for sharing!

Nice, saw this on arxiv and thought it seemed interesting, might read this as well, thanks!

Nice, thanks!

I actually wrote "The one proper method change that seems to have the biggest effect is probably adding the KoLeo regularization loss term?" in my notes, so would be nice to read more about how that works.

The main thing definitely seems to be that they scale iBOT from ViT-L/16 trained on ImageNet-22k (14 million images) to ViT-g/14 trained on their LVD-142M dataset (142 million images).

Their model distillation approach is also interesting, distilling their ViT-g down to ViT-L and smaller models.

"We revisit existing discriminative self-supervised approaches [...] such as iBOT, and we reconsider some of their design choices under the lens of a larger dataset. Most of our technical contributions are tailored toward stabilizing and accelerating [...] when scaling in model and data sizes"

iBOT: Image BERT Pre-Training with Online Tokenizer (ICLR 2022)

DINOv2: Learning Robust Visual Features without Supervision (TMLR, 2024)

DINOv2 doesn't really add much methodological difference compared to iBOT, they give a good summary of what they do:

102 papers to be exact: www.fregu856.com#service