|
|
|
|
|
|
by afranchuk
1581 days ago
|
|
|
I don't know about other people, but my tokenizer is parameterized over string slice types. So I tokenize from a slice (be it &str or ropey's RopeSlice) and some of the returned tokens have subslices. This allows tokenization to do no heap allocation. |
|
|