Y
Hacker News
new
|
ask
|
show
|
jobs
by
RandomBK
114 days ago
How well do we understand the tokenization for Claude? I'd posit that the exact human-representation of this markup is likely irrelevant if it's all being converted into a single token.
1 comments
PeterStuer
114 days ago
"<" ">" and "/>" are indeed single tokens.
link