[valine]

It’s not just the cost of training the model, it’s the cost of doing inference at scale. ChatGPT boarder line too expensive to operate already. It’s hard to imagine a larger model that both economical and used by millions of people with our current hardware.

[Jensson]

But if a larger model was good enough to replace a human, like for example a Google engineer, then it would still be worth it. So they have for sure tried to scale up, and if that extra scale gave results they would have published something about it.

Now, since the larger model wasn't good enough to replace a human engineer we can rest easy, it wont replace programmers anytime soon. If GPT-4 for example could replace engineers, OpenAI wouldn't need to monetise ChatGPT, they would just rent out artificial engineers to do coding for $10k a year.