Hacker News new | ask | show | jobs
by rsynnott 784 days ago
I mean, given they can't _count_, it would be pretty astonishing were it otherwise.
2 comments

Exactly, it’s interesting that ‘llms can’t x’, with lots of effort trying to demonstrate it, comes up so often, when we know from first principles they can’t do anything but run a Markov chain on existing words. We’ve managed to build something that is best at fooling people into thinking it can do things it can’t.
They can count occurrences of a token. Depending on the tokenization, they can't necessarily count occurrences of a character.