Hacker News new | ask | show | jobs
by matthewolfe 350 days ago
To echo the other replies, the tokenizer is definitely not the bottleneck. It just happens to be the first step in inference, so it's what I did first.