|
|
|
|
|
|
by eliben
4047 days ago
|
|
|
I guess only if you consider punctuation to be "words". A more accurate description would be "lexers work at the atom level", IOW breaking the input into atoms - the smallest non-breakable lexical units that are used in a grammar. |
|
|