[kalimanzaro]

Love the parallels people these days draw between OpenAI and Oppenheimer (ok, the Manhattan Project, but maybe thats part why OpenAI call themselves that, to alliterate)

Especially the part where Sama is trying to gather in one place the most talented, uh, anti-fas?

[Fricken]

Maybe we will look back and see it as quite timely that Nolan's biopic on Oppenheimer arrived when it did.

[fatherzine]

Manhattan Project, that can be copied and duplicated for practically free. Interesting times.

[janalsncm]

That’s not actually true at all. The basic hardware required to run even a copy of GPT-3 is outside of the budgets of all but the wealthiest companies and individuals. The engineering talent required to wire it all up is also extremely scarce.

Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.

[fatherzine]

GPT3 is 800GB, which is about as large as the largest torrent files out there. GPT4 size is unknown, so I can't comment. While I haven't run the actual numbers (life is short), assuming a GPT3 torrent, my sense is that the cost of running it would be under $1M, possibly under $100k. Compared to (hundreds of?) $B for a state-level nuclear weapons program.

But yes, technically I was wrong. It is not 'practically free', it is 'within the budget of e.g. a race boat enthusiast'.

[janalsncm]

That would be the cost of running GPT3 as-is, not to train a new model or hook it up to any live information.

GPT4 has a much larger context window (16x larger), which suggests its file size would be at least 16x as large.