"Rather than being animals that *think*, we are *animals* that think"; the last sentence of Tom Griffiths's characterisation of human intelligence through limited time, compute, and communication hits different today than it did 4 years ago.

Sometimes o1's thinking time almost feels like a slight. o1 is like "oh I thought about this uninvolved question of yours for 7 seconds and here is my 20 page essay on it"

Reviewing requires constant questioning of the motive behind your responses, every step of the way. Which btw, according to chatty, will help you become a better scientist yourself.

We squeezed all findings in Figure 1 of the preprint, but check out the paper or blogpost for details:

Paper: arxiv.org/abs/2411.12580
Blogpost: lauraruis.github.io/2024/11/10/i...

For the factual questions, the answer often shows up as highly influential, whereas for reasoning questions it does not (see bottom row of documents below).

Also, we find evidence for code being both positively and negatively influential for reasoning!

We find that a document's influence on the reasoning traces of a query is strongly predictive of that document's influence on another query with the same mathematical task, indicating that influence picks up on procedural knowledge in documents for reasoning tasks.

We find that the models rely less on individual documents when generating reasoning traces than when answering factual questions (indicated below by arrow thickness), and the set of documents they rely on is more general.

We use influence functions to estimate the effect pretraining data have on the likelihood of completions of two LLMs (7B and 35B) for factual question answering (left), and reasoning traces for simple mathematical tasks (3 tasks, one shown right).