Hacker News new | ask | show | jobs
by amelius 57 days ago
Yes, OK. But all the tutorials start with explaining how a tokenizer works. This is not necessary. And in fact makes the message of why a tokenizer is necessary not come across as well.