If you don’t have the capacity to distinguish between what’s true and what’s not, your truths are just as incidental as your lies.

Reading comprehension is an important but easily overlooked quality IMO

This is super helpful for a non-sim person, thanks for the perspective!

I agree, the paper could definitely be clearer. My assumption is “same training loop” ≈ “all else being equal”, but that can be totally incorrect.

AFAIK it's the same dataset, they just use the larger pretrained model as the teacher model. Screenshot is from the DinoV2 paper section 5: arxiv.org/abs/2304.07193

Dino-v2 is a good recent example in vision –

I wish it were only podcasts, I am seeing form steamrolling over content in academic papers more and more these days.

👋

Would like to be added!

I collected some folk knowledge for RL and stuck them in my lecture slides a couple weeks back: web.mit.edu/6.7920/www/l... See Appendix B... sorry, I know, appendix of a lecture slide deck is not the best for discovery. Suggestions very welcome.

We took a bunch of them in robot learning and made a tutorial about them! I tried to put everything that I find myself regularly telling my students there somewhere. Really think it can save some days to months of a new grad students’ life.

supervised-robot-learning.github.io