Hacker News new | ask | show | jobs
by staticautomatic 522 days ago
They’re called “lemma”
1 comments

The intent is the same, but as I understand it LLMs don't tokenize based on lemmas, though some of the tokens probably line up with them.