Yes I see. I was interested in this from your initial post: "it is clearly NOT "learning", because its ephemeral". So my Q is, does that apply to all prompt-based ("ICL" or not) methods in your view?

Well, areas that come up a lot in practise: what if I don't know or have examples of the exact task I'm going to be doing later (but I know other info that can help, such as the domain or related tasks). But now that I understand your focus I don't think it matters.

Although the Q still stands - are you "learning" from arbitrary prompts or not? If so, then it's just down to something specific about that ICL format that you are talking about.

Ok, so for you it's only ICL if it's very explicit <a few examples of performing my task (and only my task)> format? I tend to see it used a lot more generally than that now, but ok that's one definition.

That is one form of input. Does your Q still apply to the other extreme - 1M tokens or so of Wikipedia?

I've never really liked the term ICL so not sure what I think myself to be honest, but these two input types feel different somehow. Perhaps they are not though.