Y
Hacker News
new
|
ask
|
show
|
jobs
by
zurawiki
1191 days ago
In case others find this useful. Just updated for GPT-4, tiktoken-rs is a Rust crate you can use to tokenize text sent to and from OpenAI's LLM models