Hacker News new | ask | show | jobs
by wat10000 34 days ago
"飞机" and "airplane" aren't fundamentally different in terms of how they're represented to a computer. Especially for an LLM, where tokenization likely turns each of those into a single token.