I must also add that I’m assuming there’s no breakthrough architecture/pre-training/post-training method that pushes us to start everything from scratch. I’m simply asking about the decision factors in greenlighting such a full restart in the current status quo.

Given how bad I am at it, it’s out of my league too; still fun though 😅

Were you doing the NYT’s crossword? That’s how it happened for me. Also, if you want a bonus one, “doe” :)

f’ as in fine-tuned from f, not the derivative of f 😅

I got confused there yoo. Maybe something like “further condition the model’s output” (instead of update the model)?
So if the model is f(x), before the dashed line it’s f’(x), and after that it’s f(x|prompt/context).