[q1w2]

I think it's important to remember that Humans who are not-too-smart can also sound coherent, yet also babble complete nonsense.

My experience with ChatGPT thus far is that it is as intelligent as a very broadly read person who just doesn't reeeally get the complex or nuanced aspects of the content it reads - much like many real Humans.

[setgree]

Robin Hanson makes this point in better babblers http://www.overcomingbias.com/2017/03/better-babblers.html

"After eighteen years of being a professor, I’ve graded many student essays. And while I usually try to teach a deep structure of concepts, what the median student actually learns seems to mostly be a set of low order correlations. They know what words to use, which words tend to go together, which combinations tend to have positive associations, and so on. But if you ask an exam question where the deep structure answer differs from answer you’d guess looking at low order correlations, most students usually give the wrong answer."