[karaterobot]

I agree with what you say above, but my perception is that most people still view the current crop of models as a step or two away from superintelligence. That superintelligence, or AGI, is a matter of continued improvement along the current lines, rather than along entirely different lines.

[torginus]

I like to think of the 'car factory' analogy - it's populated by robots that are in some respects far superior to humans, and are doing 90% of the labor. Some ancient futurist, not having seen one before, could correctly predict that 9 out of 10 jobs will be done by robots, and arrive at the incorrect conclusion that robots have rendered humans obsolete.

In actuality, humans are still needed for the 10% the robots can't do well, or serve to enhance the productivity of humans.

I predict AI is like this and going to be for a while - it can clearly do some stuff well and sometimes better than humans, but humans will have their niches for a while.

[dekhn]

I call this the "filter changing problem". No matter how complex you make the technology, somebody still has to change the oil filter (or do whatever other maintainence is required to keep the system running). Sort of like ML-SRE, for those who are familiar with the concept.

[sseagull]

This is related to Moravec’s Paradox: https://en.wikipedia.org/wiki/Moravec%27s_paradox

“it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility”

[dekhn]

I don't think we can really say what path would lead to superintelligence (for whichever definition you desire) in the near future. Perhaps it is technically possible to achieve merely by making an embodied agent with enough different tricks in a single model (which I see as a matter of continued improvement along current lines), or maybe it requires several new things we haven't conceived yet.

Personally, my area of interest is scientific discovery. Could a model not dissimilar from what we have today, if asked a cogent question, not answer it with an experiment that could be carried out? For example, one of the most important experiments, Avery-MacCleod, which proved (to the extent that you can prove anything in biology) that DNA, not protein, was the primary element of heredity, is not all that complicated, and the mechanical details seem nearly in reach of modern ML techniques. Similarly, could the ML model provide a significant advance in the area of understanding the molecular function in intimate detail of proteins as determined by their structure (which AlphaFold does not do, yet), complete with experimental instructions on how to verify these hypotheses? As of this time, my review of modern ML methods for science suggest we have made some advances, but still have not passed the "phase transition" demonstrating superscientist-level understanding of any field. But perhaps it will just fall out naturally from improved methods for media generation/parsing and ad targeting.

I continue to remain hopeful that within my remaining 20-40 or so years (I'm a typical american male, age 51, with a genome that contains no known risk factors) I will see something like what Vinge describes in https://edoras.sdsu.edu/~vinge/misc/singularity.html in a way that is demonstrable and safe, but honestly, I think it could go in any number of directions from "grim meat-hook future" to "unexpected asteroid takes out human life on the planet, leaving tardigrades to inherit the earth" to "kardyshev-scale civilization".

[tim333]

Thanks for the link to Vinge's thing - I hadn't read that. Shame he didn't get the superintelligence within thirty years of 1993 and died in 2024. Still soonish. I've always been interested in physics and have a hunch that we can't figure things like quantum gravity due to mind limitations. If an AI could do that that would really be superintelligence.