[hxugufjfjf]

I think your two final sentences capture the essence of what I was going to respond. "It doesn't matter if the answer is convincing-looking and wrong. It needs to work / be syntactically correct at a minimum, which OpenAI seems good at. However, the OP and others needs to test and evaluate if the proposed answer solve the original problem. And if its not, it will quickly be revealed as such, and "downvoted" or whatever stackoverflow functionality exists to indicate bad answers. This applies to both human and AI-generated answers."

[scubbo]

Yeah, absolutely. My position on AGIs for a long long time has been that they're great tools for a) generating insights into a large amount of data very quickly, and b) generating new instances of <thing> from examples of <thing> to help with exploring the possibility space of <thing>; but that any output or conclusion they generate _must_ be checked by a Human In The Loop, or at the very least their actions must be reversible without damage in the case of error.

[troupe]

Good point. It isn't like it is hard to find wrong answers on SO that were written by (hopefully) well meaning humans.