(If you're interested, a truly random set of 50 numbers might look like this.)

/end

Humans are also famously bad at generating random numbers. But we err in quite a different way than the LLM does.

Maybe the human style of "random" number generation is something that never really manifests itself in anything written and so the LLM can't pick up on it?

5/N

When I started investigating this I imagined there would be a signature, but I thought it would be a far more subtle statistical thing.

4/N

The most interesting result comes from Deepseek R1, where the internal monologue shows that it *knows* it isn't doing it right.

After an agonizing debate with itself and a lot of heartrending effort at making it more random it eventually ends up with:

3/N

On the other hand, if you're an LLM, apparently the distribution will look like this, with each digit appearing *exactly* as often as the others.

(This isn't random at all!)

I get identical results for ChatGPT, Claude and Deepseek. 2/N