Hacker News new | ask | show | jobs
by msp26 639 days ago
> If your knowledge base is smaller than 200,000 tokens (about 500 pages of material)

I would prefer that anthropic just release their tokeniser so we don't have to make guesses.

1 comments

shouldn't this be possible to reverse-engineer since they stream the responses token-by-token?