A pity, as I used to hold the one well known person in high regard, which makes it all the more disappointing now. It was about whether LLMs can count letters. I provided a working demo and peer-reviewed research showing it’s possible under the right setup, but the exchange ended before it began.

And read tip: "Can ChatGPT Learn to Count Letters?" Mar. 2025, pp. 96-99, vol. 58 DOI Bookmark: 10.1109/MC.2024.3488313

... and with the right tokenizer/configuration it can effectively reconstruct letters before reasoning over them. So, while the standard LLM experience might fail on this task, it’s not an inherent impossibility of LLMs themselves. It’s an implementation detail.

... I can feed an unknown fantasy word, have it break it down into single letters, and count them correctly, as shown in the previous post. This works because the model has learned statistical associations between tokens and their underlying character sequences, ...

Sean, you are correct that most token-based LLMs don’t literally "see" letters and operate on subword tokens, but this is not a hard limitation of all LLM setups. With the Qwen3 30B A3B model running in my own WIP inference engine (Object Pascal + x86-64 inline assembler), ...

At least the Qwen3 30B A3B model can break an unknown fantasy word down from a set of tokens into single letters and count them correctly, as shown in the attached screenshot (running in my own WIP LLM inferencing engine, implemented in Object Pascal and x86-64 inline assembler).

🎧 Spotify: open.spotify.com/intl-de/albu...

🍎 Apple Music: music.apple.com/de/album/end...

📻 Deezer: www.deezer.com/en/album/711...

🛒 Amazon: www.amazon.de/dp/B0DWT1F4C6

🌊 Tidal: tidal.com/browse/track...

🎵 YouTube Music: www.youtube.com/watch?v=9FOI...