That’s not actually true at all. The basic hardware required to run even a copy of GPT-3 is outside of the budgets of all but the wealthiest companies and individuals. The engineering talent required to wire it all up is also extremely scarce.
Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.
GPT3 is 800GB, which is about as large as the largest torrent files out there. GPT4 size is unknown, so I can't comment. While I haven't run the actual numbers (life is short), assuming a GPT3 torrent, my sense is that the cost of running it would be under $1M, possibly under $100k. Compared to (hundreds of?) $B for a state-level nuclear weapons program.
But yes, technically I was wrong. It is not 'practically free', it is 'within the budget of e.g. a race boat enthusiast'.
Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.