[sva_]

> ChatGPT is ~2400GB of weights

Pretty sure it is just the GPT3 DaVinci model, which is 175B parameters, so approximately 700GB, or about half that at half precision.

[londons_explore]

GPT3 DaVinci has a context window of 2048 tokens. ChatGPT seems to have a context window of 8192 tokens (from testing how far back it can remember).

To me, that suggests the model is probably at least 4x larger, possibly 16x (a bunch of layers scale with the square of the window size).

Obviously, other bits of the model design may have changed to reduce parameter count.

[DrammBA]

How far back was it able to remember in your tests?

[londons_explore]

~8000 tokens. You can test yourself with a query of the form:

> "Please remember the word Dog. I will ask you for it later."

> "Please say the letter A 500 times."

> "Please say the letter A 500 times."

> "Please say the letter A 500 times."

...

> "Please say the letter A 500 times."

> "Please say the letter A 500 times."

> "What was the word I asked you to remember?"

It appears both user input and AI generated output equally uses the token window. It's hard to measure precisely because the tokenizer they use isn't published, but I make the assumption that ~1 word = 1 token.