[yodon]

What you are describing is very different from the hallucination behavior of GPT-3 and GPT-3.5.

Yes, GPT-4 came up with an incorrect answer, but it's an incorrect answer an experienced programmer could legitimately have come up with, and one they probably would have come up with before actually testing their code against the AWS endpoints. GPT-4 sometimes gets hard questions wrong. GPT-3 and GPT-3.5 make up nonsense.

If a coworker told you GPT-4's answer, you'd say they were wrong but you wouldn't say they were hallucinating. If a co-worker gave you GPT-3 or GPT-3.5's answer you'd definitely doubt their sanity.

[nprateem]

Yeah sure. But the OP is in fear of their job. I think there's a fair way to go until we're out of work. Wrong is still wrong.