Y
Hacker News
new
|
ask
|
show
|
jobs
by
wolfgarbe
559 days ago
Currently, you can choose between tokenizers with or without folding. But configurability per language or full customizability of the folding logic by the user is a good idea.