Y
Hacker News
new
|
ask
|
show
|
jobs
by
senseiV
875 days ago
GPT 2 and 3 used the p50K right? Then GPT-4 used cl100K
1 comments
lifthrasiir
875 days ago
Yeah, see [1].
[1]
https://github.com/openai/tiktoken/blob/main/tiktoken/model....
link
[1] https://github.com/openai/tiktoken/blob/main/tiktoken/model....