[nwhitehead]

You would think so...

I asked this question in a college-level class with clickers. For the initial question I told them, "This is a trick question, your first answer might not be right". Still less than 10% of students got the right answer.

[adamisom]

Students are more arrogant than LLMs (but still more generally intelligent)

[roenxi]

This idea that students are "more generally intelligent" requires specific arguments to support. ChatGPT alone has a vast breadth of knowledge that is impossible for a human to keep up with as well as superior skills to a student in any number of fields. I can find evidence that it has an IQ of around 124 [0] which is going to stretch most students (although in fairness the same article also speculates an IQ of 0).

Students can keep ahead of it with training in specific fields and it has a few weaknesses in specific skills but I think someone could make a reasonable claim that ChatGPT has superior general intelligence.

[0] https://medium.com/@soltrinox/the-i-q-of-gpt4-is-124-approx-...

[chmod775]

> superior skills to a student in any number of fields

It's abhorrent any time we measure pure distilled intelligence.

When asked to come up with any non-basic novel algorithm and data structure, it creates nonsense.

Especially when you ask it to create vector instruction friendly memory layouts and it can't code in its preferred way. I had some fun trying to make it spit out a brute-force-ish solver for a problem involving basic orbital mechanics and some forces. Wouldn't even want try something more complicated. It can do generalized solvers somewhat, since it can copy that homework, but none that can express the kinds of terms you'd be working with (despite those also having code available in some research papers).

Speaking of which, it cannot even figure out some basic truths in orbital mechanics that can be somewhat easily derived from the formulas commonly given, nine times out of ten (you can get there if you're very patient and are able to filter its wrong answers).

But at the end of the day it was still a valuable tool to me as I was learning these things myself, since despite being often wrong, it nevertheless spat out useful things I could plug into Google to find more trustworthy sources that would teach me. Really neat if you're going in blind into a new subject.

[magic_hamster]

Claiming that Chatgpt is more intelligent than a student is the same as saying an encyclopedia or a library is more intelligent than a student. Sure they retain more information. But Chatgpt is not AGI and it has no idea what it is even talking about.

[ben_w]

> I can find evidence that it has an IQ of around 124

Despite me being someone who is generally impressed by the best LLMs, I think this says more about IQ tests than it does any AI.

Which isn't to shame those tests — we made those tests for humans, we were the only intelligence we knew of that used complex abstract language and tools until about InstructGPT — but it does mean we should reconsider what we mean by "intelligence".

My gut feeling is that a better measure is how fast we learn stuff. Computers have a speed advantage because transistors outpace synapses by the degree to which a pack of wolves outpaces continental drift (yes I did do the calculation), so what I mean here is how many examples rather than how many seconds.

But as I say, gut feeling — this isn't a detailed proposal for a new measure, it likely needs a lot of work to even turn this into something that can be a good measure.

[deafpolygon]

Knowledge != Intelligence