Hacker News new | ask | show | jobs
by strbean 305 days ago
We're up to a gazillion parameters already, maybe the next step is to just ditch the tokenization step and let the LLMs encode the tokenization process internally?