|
|
|
|
|
|
by WilliamBerglund
1119 days ago
|
|
|
Hello (author here), I found the OpenAI tool (https://platform.openai.com/tokenizer
) for visualising tokens really useful when working on token optimisation strategies (and other token related tasks). But unfortunately that site doesn't support the tokeniser that GPT4 uses (currently). So I threw together a stackblitz that uses the js-tiktoken to do the same thing. Hopefully it will be useful for someone else too! Try it here: https://react-ts-h9yrjh.stackblitz.io/ |
|
|