[short_sells_poo]

This is not even about capabilities but responsibility. In an open source context where the maintainers take no responsibility for the code, it's perhaps easier. In a professional context, ultimately it's the human who is responsible, and the human has to make the call whether they trust the LLM enough.

Imagine someone vibe codes the code for a radiotherapy machine and it fries a patient (humans have made these errors). The developer won't be able to point to OpenAI and blame them for this, the developer is personally responsible for this (well, their employer is most likely). Ergo, in any setting where there is significant monetary or health risk at stake, humans have to review the code at least to show that they've done their due diligence.

I'm sure we are going to have some epic cases around someone messing up this way.