Hacker News new | ask | show | jobs
by AndrewKemendo 3034 days ago
So, AlphaZero doesn't mean we're closer to _general_ AI. Quite the contrary: it's a very specialised form of AI that will be very difficult to use in any different task than chess, shoggi or go.

This is a very true statement and one that I think a lot of people who aren't in ML/DL, but are "worried" about AGI, miss.

There is however a common thread with everyone in AI, that they tend to think of AGI as "One algorithm to rule them all."

As a practitioner and AGI researcher however I think that AGI is more of a system of specialized or narrow AI tasks that can together solve all tasks. At the risk of oversimplifying and anthropomorphizing, this type of problem solving is functionally how we do it as humans.

So having a corpus of solved narrow systems (discrete known rule space in the sense of AlphaGo etc...) that is "activated" by an executive function which can recognize the problem set and then pass subsets of a larger problem to the narrow solutions. Those solutions are then "backpropagated" and synthesized into the general problem solution.

In that sense, I would argue that narrow solutions like AlphaGo etc... do get us closer to General AI because they grow the corpus of solution paths for the general problems.

4 comments

I think you are only thinking of supervised learning's capabilities, which I assume is the field of your AGI research? I'm working with reinforcement learning and DRL research, and RL was born to address the short coming of supervised learning. DeepMind is arguably the forerunner in RL right now, and AlphaZero is the crystalline of their RL research.

Yes, AlphaGo uses NN and other SL techniques, but the core is very much DQN based RL. No amount of SL can effectively play go and invent new moves. RL can already solve a large number of real world problems with a rather simple algorithm, from self driving cars to video games to NLP. RL can tackle all those problems with pretty much the same core algorithm. The question lies less in IF RL can solve more general AI problems, but rather HOW to solve it. From a high level view, we are having a lot of trouble with its convergence properties mathematically and its extreme sample inefficiency. This is the reason why Boston Dynamics doesn't use much RL, Waymo doesn't use much RL, simply because they can do much better with current techniques without going RL.

AlphaGo is still a major step forward regardless, because it's one of the biggest leap in RL we've taken in the recent years. It suddenly lets RL stably converge on solutions more than we could ever before. AlphaGo's contribution is more than just that it built a specialized Go bot, but rather a much more stable RL algorithm that lets us approximate non linear functions (majority of the real world applications are non linear). If I were to put my money, AI could very well be entering a new era with AlphaGo and their DQN.

I disagree. The naming convention [Artificial Intelligence] still is a large shoe that these purpose built applied engineering solutions have yet to fill. Meanwhile, for profit/notoriety/marketing people want to trample on yet another name space? What you just described is essentially the architecture of a self-driving car. It's yet another applied engineering solution of Artificial Intelligence. It is not Artificial General Intelligence. Scaling/Distributing the computational space of an applied Artificial Intelligence solution is not Artificial General Intelligence. This is the same thing that lead to optimization algorithms being called Artificial Intelligence. If you aren't able to maintain foundational distinguishment, you lose track of what you're searching for and trying to achieve. Outwardly, you capture more money and attention. Inwardly, you become unraveled and lose your capability to solve the elusive problem. Eventually after much fame, wealth, and feigned 'success', one asks themselves : Was it worth it? Depends on what your original aim was.
I'm not sure what you're arguing but it seems like my key point wasn't communicated well.

What you just described is essentially the architecture of a self-driving car.

Yes, every narrow AI is a system of systems to an extent. So expand on that concept but outside of a single firm/system. Such that the self driving car system is one single solution path solving "transportation" which would comprise automated flight/rail etc... and is a node in a larger general system - like hub and spoke.

The naming convention [Artificial Intelligence] still is a large shoe that these purpose built applied engineering solutions have yet to fill.

Nobody is questioning that. The size of the narrow AI market is arguably infinite.

You seem to be arguing that a single entity will fail if it attempts to take a narrow AI system and make it generalizable. Of which I am in agreement with.

If however there were 10,000 or 100,000 or 1,000,000 narrow AI companies/systems (like a self driving car system or alphago etc...) those could fill the corpus of solutions which an executive function system could utilize depending on the application and together they would be what we call AGI.

> I'm not sure what you're arguing but it seems like my key point wasn't communicated well.

It was and quite well. Were speaking the same language. We just have different conclusions.

> Yes, every narrow AI is a system of systems to an extent. So expand on that concept but outside of a single firm/system. Such that the self driving car system is one single solution path solving "transportation" which would comprise automated flight/rail etc... and is a node in a larger general system - like hub and spoke.

And you still have nothing more than a hub and spoke system of systems authored for specific problems spaces and you're spokes will increase with every new problem space until you overwhelm your hub. A horrible architectural approach that if not caught in the initial stages will result in catastrophe down the road... Weak AI is weak AI no matter how you scale it.

> You seem to be arguing that a single entity will fail if it attempts to take a narrow AI system and make it generalizable. Of which I am in agreement with.

This is a start in the right direction...

> If however there were 10,000 or 100,000 or 1,000,000 narrow AI companies/systems (like a self driving car system or alphago etc...) those could fill the corpus of solutions which an executive function system could utilize depending on the application and together they would be what we call AGI.

No, its strung together weak AI. It will require significantly and unreasonable amounts of resources. Its capability will increasingly reach diminishing returns and you'll end up with a frankenstein monster code base that no one can manage or understand.. Sounds a lot like the path Weak AI is already heading down.. At such a point, it's best to just scrap it and start all over. Something that Hinton and other prominent figures are finally admitting. Something I concluded year ago which lead me down a different path. Now, you're more than welcome to state : Well hey man that's your opinion and you're wrong and I'll wish the 10s,100s, million of narrow AI companies the best just as was conveyed to me a umber of years ago. Weak AI is Weak AI. It is a class of optimization algorithms. You can jerry rig this all you want.. You still have nothing more than a system of systems of optimization algos. If you think this is what intelligence is, I'm not sure what to say.

You still have nothing more than a system of systems of optimization algos. If you think this is what intelligence is, I'm not sure what to say.

Until someone comes up with a better definition of intelligence that's what I'm sticking with. I think you're looking for an elegant solution right out of the box - the "one algorithm to rule them all" and I don't think that is feasible from an engineering perspective if for no other reason than no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

Having raised three other humans and observing them while building DL systems myself for a living, I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function. AGI is as much a community building and humanity wide input collection challenge as it is a math problem. We need to think about it that way.

I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function.

I’m in complete agreement with this. I try to avoid AGI discussions because people get upset when I argue that the vast majority of human “intelligence” seems to be strong pattern matching, and we can’t really define the parts that aren’t in any useful way.

Take the person you are discussing this with. The majority of their point seems to be a hang-up on the word “intelligence”.

I find that a pointless thing to argue over. Just agree and say it is an intelligence simulator which is indistinguishable from a real intelligence.

> Until someone comes up with a better definition of intelligence that's what I'm sticking with.

You'll get a capability demo instead. It wont fail to impress. Definitions and designs are for another day.

> I think you're looking for an elegant solution right out of the box - the "one algorithm to rule them all" and I don't think that is feasible from an engineering perspective if for no other reason than no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

What else is one looking for who claims they're trying to solve the Intelligence problem? Marketing an optimization algorithm as the next coming might make you rich in the short term but it doesn't bring you closer to the truth. It does in fact take your further away. So, 'the elegant solution'/'the hard problem' was the only thing I set out to tackle some years ago. Otherwise, i'd have been wasting my time/not being truthful with myself. It's feasible from a research and engineering perspective. Few commit themselves to the TRUE task and the likelihood of failure. I was ok with that it and stuck with it. I self-funded my work. It mainly centered on research. Thus, there were no exits. I either saw it through and achieve it or I didn't.

As far as :

> no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

Sure it does. Look in the mirror and log onto the web. I've let the misses play online for a bit now ;).

> Having raised three other humans and observing them while building DL systems myself for a living, I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function. AGI is as much a community building and humanity wide input collection challenge as it is a math problem. We need to think about it that way.

My graduate work centered on the underpinnings of DL (Distributed Optimization). After years of industry experience, I searched for a new challenge. After some open ended research in physics/photonics, I came to Artificial Intelligence. I scratched my head for 3-4 months as to why (Distributed Optimization) was being called Artificial Intelligence. I took the broad lot of it and threw it in the trash as prominent figures are only now stating : https://www.axios.com/artificial-intelligence-pioneer-says-w...

You're thinking about AGI as if its a chain of DL systems because that's what's made you money and where your work has centered on over the years. I took the broad majority and trashed it as Hinton now indicated others should do and started from scratch. I have no such bias. However, as my graduate work centered on the fundamental underpinnings of statistical optimization / distributed optimization, I know exactly what its limits are.

The human race is far more than a hodgepodge of optimization algos w/ an executive function (whatever that might be given the clearly varied forms of it).

Pardon my question: what kind of education did you have?

I deeply regret not studying Computer Science but yours seems to be deeper than that. I'd be very interested in the courses you took.

Apologies for the off-topic question but I am simply too tempted not to ask them, I admit:

How does one make a living out of being an AGI researcher? How does one pay mortgage and has money and freedom to go on impulsive vacations with their wife?

And what does an AGI practitioner even mean?

How does one make a living out of being an AGI researcher? How does one pay mortgage and has money and freedom to go on impulsive vacations with their wife?

Well you don't, unless you work for OpenAI (which I don't). Not sure how OpenAI does it or how well they pay but I'm sure it's good. They have good donors.

And what does an AGI practitioner even mean?

To clarify, I'm a ML practitioner. AGI practitioner means nothing

I don't mean to sound demeaning but yes, IMO "AGI practitioner" indeed means nothing (and I am saying that as a guy who aspires to work on AGI). Many of us programmers have interesting ideas in the area of AGI but quite frankly, we can "practice" them our whole lives without achieving squat.

Thank you for responding, much appreciated.

Well - SOAR.
What does this comment mean?
The point of soar was/is to provide a framework for a collection of problem solvers with diverse methods to work together in a single agent. This was developed because of the insight that humans use diverse strategies to negotiate the challenges of their environment. Or... Lots of weak Ai adding up fyi general Ai.
Oh yes, of course.