Ya maybe. I still think it's cool. E.g. you see experts specialized for groups of imagenet categories in vision MoEs. arxiv.org/abs/2106.05974