[madiator]

Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.

[joshka]

https://www.reddit.com/r/ChatGPT/comments/zhzjpq/comment/izo...

>It's around 500gbs and requires around 300+gbs of vram from my understanding and runs on one of the largest super computers in the world. Sable diffusion has around 6 billion parameters gpt-3/chatgpt has 175 billion.

[cal85]

Wouldn’t that be possible with about 4 powerful GPUs? Or does it not work like that?

[taocoyote]

Possibly, but that would be 10 of thousands of dollars worth of GPUs.

[kurtoid]

Silly question: how does OpenAI host/serve it?

[magixx]

I think on professional hardware you can get 80G of memory per GPU and they can likely do memory pooling.