[stevekemp]

It's basically a clone of the tokenizer from this python implementation https://norvig.com/lispy.html which was referenced in the introduction.

The same code has been seen in a bunch of simple/similar lisp interpreters in different languages.

To be honest I'm not sure if that was the first documented tokenizer using this simple approach, but it is definitely a common pattern for such things - and as you say it is naive at best, and buggy at worst. That said it is easy to get it working, and later fix it properly.