[Nevermark]

You keep saying LLMs. But models across many modalities are rapidly improving, evolving and merging.

Not sure what you mean by "hand waving" or "fantastical nonsense". Maybe just address points I actually make.

1. Machines with greater cognitive ability than us, potentially much greater, would be an economic challenge for human beings. With no clear answer as to how humans could manage that challenge.

2. Machines are getting more capable, year-to-year, faster than any human can or ever will improve. With no signs of slowing, or any areas where they are failing to improve.

3. None of this is new. Computing capabilities have compounded steadily since the first transistors less than a century ago. Human's biologically driven cognition, in contrast, has not improved.

4. Machine capabilities are now regularly passing us in new areas, and rapidly approaching the general threshold noted above.

Explain your perspective, I am genuinely interested.

For instance, what cognitive capability do you have, that you believe machines won't exceed within 5 years.

[keiferski]

1. Possibly, but not necessarily. Many jobs are not dependent on intelligence, and more intelligence isn't going to make much of a difference. Huge portions of the economy have absolutely nothing to do with intelligence and AI bots won't make a difference. For rote, white-collar style work, yes, it's already making a huge impact. But that is certainly not extendable to all human work, even all white collar work.

2-4. True, but this doesn't mean that things will exponentially increase indefinitely. I use LLMs constantly and find the idea absurd that they're approaching human-level intelligence. They make basic, common mistakes that any average person would never make. There may be simply structural reasons why the progress you (and sci-fi novels) imagine is not possible.

My critique is more like:

1. You cannot just assume exponential growth will continue forever.

2. Current LLMs are powerful but in no way does this indicate that it's just a matter of time until we have AGI - itself a pretty vague marketing term.

3. The main reason, which is: human beings are reactive. You can already see in the market that openly using AI is becoming a market disadvantage in many sectors. No one wants to read a blog written by ChatGPT, and slop-coded SaaS apps aren't actually that competitive with serious companies run by professionals.

As AI stuff supposedly "replaces" human work, that human work will adjust and become more creative. This process has already kind of happened with writing – initially it was assumed that copywriters would all lose their jobs. But it turns out that AI-speak is predictable and easily discovered, and the choice of what to write in the first place is actually far more important. Companies are still hiring writers, with the added caveat that knowing when and how to use AI is a new skill they have to learn.

Adding to this is the fact that many human solutions to things are the result of being alive, being embedded in reality. Unlikely for this (a robot being so lifelike and embodied as to replicate human experience, a la Blade Runner) to happen in the next century or two.

If your response to all of this is, "well sci-fi novels predict that humanoid robots will be better at humans than everything" then that isn't a serious opinion, I'm sorry.

I find it far more likely that a "cyborg" role outcompetes any AI-only one, now and in the future.