yeah, i think the main reason is that larger models generalize better and can be distilled. this seems like a satisfying enough reason?

i really hope every ML class can catch up to teaching this (just yesterday i also wrote a post about how double descent appears in linear regression, specifically due to the inductive bias of minimum norm solutions kothasuhas.github.io/writing/doub...)