[nicklecompte]

I wonder if GPT-4's performance has degraded in recent months because there are less human data contractors on standby to answer questions GPT flags as low-confidence. GPT might be "refusing to answer questions" because it's not able to escalate tough queries to a human.

[shawabawa3]

Not plausible, even when it's on slow mode it's too fast to be contacted out to humans

[nicklecompte]

To be clear: ChatGPT-4 is in general both far too fast and far too stupid for humans to be answering any more than a tiny fraction (<< 1%) of the queries.

But last year I repeatedly saw ChatGPT-4 respond token-by-token much more slowly than a human would! E.g. several seconds between words. It was clearly not a human responding: at least a few times I was testing on preschool counting questions and GPT-4 was not able to answer them. I interpreted the slowness as GPT's poor quantitative reasoning. But what you're saying is simply not true, sometimes ChatGPT-4 is (or was) extremely slow.

Regardless, if OpenAI was running this con it probably wouldn't have been real-time humans writing. First of all it might be enough to have a human in the "mixture of experts" who decides the best of multiple responses when GPT-4 is unable to come to an automated conclusion. But humans could be writing ChatGPT responses due to a quirk in their UX:

- ChatGPT errors out on a certain question and asks you to try again later, as it does (or used to do) frequently

- the response is prepared by the human contractor while the user waits patiently for ChatGPT to resolve its technical difficulty

- when the user asks again ChatGPT can largely read off the answer, using its (genuine) language-processing abilities to handle variations in phrasing/etc