Hacker News new | ask | show | jobs
by Boojum333 1087 days ago
Downloaded it's very responsive and the ability to edit contexts is really useful. I was curious as how you are able to calculate the number of prompt tokens in real time. Did you implement a method for Byte Pair Encoding yourself and how did you manage to match it with OpenAI's implementation for ChatGPT?