Ask HN: AI is smarter than the average person but feels like nothing's changed?

[geepytee]

Primarily asking because I'd like to hear different perspectives, perhaps I am missing something.

Arguably, since OpenAI released the o1 models, LLMs are now 'smarter' than the average human when measured by IQ (I'm going by this study [0] which sets o1 at a 120 IQ).

What I am trying to wrap my head around is why has this not changed our entire world much? Sure, if you live on Twitter, a lot of people made a big deal about it. But in my day to day, specially when offline, nothing seems to have changed. In fact, I don't think most people are even aware that a computer is now smarter and cheaper than them and it's widely available via API.

Am I exaggerating things here? It almost feels like the world has not caught up to the latest technology. Does this happen with every new tech? Is this period basically a huge opportunity for early adopters? Perhaps we are missing ways to connect the o1 brain to the real world so it can have real world applications?

For context, I am deep in LLMs stuff daily as it is part of my work. I am keenly aware of the improvements that have been made in coding for example, I just don't believe this is on the same magnitude as 'AI is now smarter than the average human'.

The other side of this argument is that the LLMs are not that good, and they just test high because the questions are part of the training data, and in fact they cannot adapt and learn on the spot the way humans can (which I believe is the point of the ARC prize [1]). Another counter-argument might be that it's just too early?

Would love to hear what you have to say. Tell me how I'm wrong, or tell me how you think AI has already materially changed our world in a big way.

[0] - https://trackingai.org/IQ

[1] - https://arcprize.org/

[mikequinlan]

1. IQ is a bad measurement of intelligence.

2. IQ is a quotient. What age did they say that o1 was?

3. Using a Mensa test (Mensa Norway apparently) is a bad way to determine IQ.

https://test.mensa.no/home/test/en

"This online test gives an indication of general cognitive abilities, represented by an IQ score of between 85 and 145, where 100 is the population average. This test is not a substitute for professional intelligence tests, such as those administered by Mensa and licensed psychologists.

This test consists of 35 puzzles in the form of visual patterns that must be solved within a 25-minute time limit. Participation requires neither specialised knowledge nor mathematical skills. The puzzles, which get progressively difficult, are weighted equally, so you get a point for each correct answer. You do not get bonus points for finishing the test early, so try to manage your time optimally. Also, you are not penalised for answering incorrectly, so make a guess whenever you are unsure."

[zifpanachr23]

Surely at some point higher IQ (in the IQ test sense) must also be likely to reach a point of diminishing or even negative returns.

I'm unconvinced that if everybody suddenly woke up tomorrow with an IQ score that was 20 points higher, that all of the most pressing issues in the world would start to be solved. Maybe some would, but I suspect we'd find out that there are tradeoffs that occur when we min/max a particular form of intelligence. There are also raw physical limits to contend with. The speed of light will remain a limit no matter how high your IQ. So there may be diminishing returns to science as well.

There are all sorts of ethical and social and game theory related problems that don't seem like they would be at all helped out by having a radically more analytically competent populace. If anything, I suspect we may be worse off.

[seccode]

Agree that IQ is a bad test for intelligence, speaking as someone with a very high IQ. I am interested to know your thoughts about the test for AI that I came up with (check my most recent post). Let me know what you think!

[rsynnott]

> Arguably, since OpenAI released the o1 models, LLMs are now 'smarter' than the average human when measured by IQ

Arguably, pigs can fly. It's not a _good_ argument, but it's an argument, I suppose.

No, of course LLMs aren't smarter than the average person, don't be silly.

(Twist: The OP was written by a self-aggrandising robot.)

[JohnFen]

AI is clearly not smarter than the majority of people, let alone the average person.

The IQ test thing is, in my opinion, not significant for a couple of important reasons. First is that it's an open question as to how much IQ correlates with intelligence (particularly since we still don't have a solid definition of what "intelligence" is), and the second is that LLMs certainly ingested many IQ tests and the answers into their training data.

[ungreased0675]

Current models aren’t accurate, reliable and break in unpredictable ways. Therefore it’s hard to rely on them for serious tasks.

I also wouldn’t classify GPT4 as intelligent the same way I wouldn’t classify Google search as intelligent. It’s a software tool.

[fuzzfactor]

I would say there is some chance that the world is not impressed since so many people have thought computers they already had for a while were more advanced than this when they bought them :\

A lot of times otherwise they wouldn't have bought them !

[zifpanachr23]

They aren't embodied, they don't experience the passage of time, or the physical world, or emotion in the way a human does.

I'm also just not sure how ultimately useful more powerful analytical tools will be. Most of the issues in the world are either physical issues and involve resources constraints in such a way that the ephemeral nature of AI is ill suited to handle. Or they are social issues, in which case the lack of wisdom and authentic human experience also makes AI ill suited to the tasks.

The primary conceit of the current emphasis on digital technology as a means to solve our problems is likely the idea that if we were just smart enough and thought hard enough or had an ultra intelligent assistant, intelligence would improve things and make the world a better place. That may help up to a certain point, but it doesn't seem obvious to me that is will continue to have positive returns when taken to it's logical extreme. There's also the issue of meaning and goals and teleology. I think that complicates a lot of the stories told by the more philosophically minded proponents of AI. In other words, suppose AI provided us a means to realize some kind of post-scarcity society...what's the point? What next? We are all familiar with the Rat Park experiments. Even in the most fantastically successful version of the future where AI massively surpasses the expectations of even the most dedicated and convinced proponents...that doesn't help us deal with the more fundamental issues. The industrial revolution already probably took us about as far down the path towards "post scarcity" that human psychology will allow. And the internet probably pushed us over the edge to the extent that it may engender the emergence of more serious reactionary impulses, and diminishing returns in the economic sense as it weakens us socially and emotionally and spiritually.

It doesn't seem obvious to me that intelligence (in the IQ test sense, or the LLM regurgitating decent responses to a multiple choice test sense) is necessarily always going to be socially, culturally, or evolutionarily adaptive. There is probably some point at which higher analytical intelligence begins to have diminishing returns, and some point after that where it even becomes maladaptive for most people in most contexts.

There's also the issue of trust. AI isn't trusted. And the tech community more broadly but especially the "AI community" portion of that is especially not trusted. And the tools aren't really that exciting to people outside of the tech social sphere. And that isn't even to mention the negative potential effects with respect to the issue of social trust between real people (did they really write that and mean it, or did an AI do it for them? Can I trust that these people on HackerNews, that I've just spent 20 minutes writing a heartfelt take on a serious issue for, are even real and not AI bots?).

So besides all my high minded speculation above about the nature and value of the specific kind of intelligence AI vendors primarily claim their products to possess...you've also just got a plain old fashioned product problem. People aren't really all that excited about the product and don't see how it could realistically solve the issues they face in their day to day lives. That seems like a very reasonable take to me and probably the most obvious reason why AI hasn't (and maybe won't) radically change the way we live our lives. I'm mostly involved in systems engineering and infrastructure work and consulting on those topics, and I'm kinda in the same position the rest of the population outside of Big Silicon Valley Web Dev Tech land finds themselves in. I've yet to find a killer application for the technology. I'll subscribe for a month to Anthropic or OpenAI when there is a big new release to give them a shot. But I've yet to renew over the past year or so because I just don't find them very useful for my work.

TLDR: I think AI developers are way out on their skis and living in a bubble and are optimizing for the wrong things. I don't really believe in "AI doom", but I think a lot of the pro safety people were really just trying to address this more boring and less dramatic social and product problem in the only way they know how (via developing strange new religious eschatology and other related hobbies of people that have been hanging out in these deeply anti social groups for so long). I'm sympathetic to their efforts (when they are legitimate and not marketing). But I'm not sure the prognostications of doom are all that useful at this point, most people just get bad vibes from AI and don't like it and don't see the point and don't really need much convincing in that respect. The products and the marketing that surround them are pretty on the knose about being anti-social and anti-human, and ordinary people have done a good job picking up on those vibes.

[JohnFen]

> The products and the marketing that surround them are pretty on the knose about being anti-social and anti-human

This is what terrifies me about "AI". It's not the technology itself, it's that the companies and most of the vocal proponents are very clearly sociopathic.