[YeGoblynQueenne]

About AlphaZero particularly, a few things must be kept in mind.

First, AlphaZero still makes use of a Monte Carlo Tree Search algorithm to search for good moves. MCTS is a powerful algorithm with a very limited scope: zero-sum, perfect information games. So for instance, it would be very difficult to see how to use MCTS-based AlphaZero in, e.g., training self-driving cars.

Second, the AlphaZero architecture is precisely mapped onto a checkerboard and will not learn anything about games that don't use a checkerboard, or any situation that is not possible to model as a game played on a checkerboard.

Third, the AlphaZero architecture is also precisely mapped onto the range of moves of pieces in chess, shoggi and go. Again, AlphaZero would be useless in any game that used pieces with different moves (e.g. a piece with a zig-zag move, or a piece allowed to move in spirals etc).

All of the above of course can be mitigated with different architectural choices, but to make those choices, implement them and validate them will take a great deal of time.

So, AlphaZero doesn't mean we're closer to _general_ AI. Quite the contrary: it's a very specialised form of AI that will be very difficult to use in any different task than chess, shoggi or go.

[AndrewKemendo]

So, AlphaZero doesn't mean we're closer to _general_ AI. Quite the contrary: it's a very specialised form of AI that will be very difficult to use in any different task than chess, shoggi or go.

This is a very true statement and one that I think a lot of people who aren't in ML/DL, but are "worried" about AGI, miss.

There is however a common thread with everyone in AI, that they tend to think of AGI as "One algorithm to rule them all."

As a practitioner and AGI researcher however I think that AGI is more of a system of specialized or narrow AI tasks that can together solve all tasks. At the risk of oversimplifying and anthropomorphizing, this type of problem solving is functionally how we do it as humans.

So having a corpus of solved narrow systems (discrete known rule space in the sense of AlphaGo etc...) that is "activated" by an executive function which can recognize the problem set and then pass subsets of a larger problem to the narrow solutions. Those solutions are then "backpropagated" and synthesized into the general problem solution.

In that sense, I would argue that narrow solutions like AlphaGo etc... do get us closer to General AI because they grow the corpus of solution paths for the general problems.

[yonkshi]

I think you are only thinking of supervised learning's capabilities, which I assume is the field of your AGI research? I'm working with reinforcement learning and DRL research, and RL was born to address the short coming of supervised learning. DeepMind is arguably the forerunner in RL right now, and AlphaZero is the crystalline of their RL research.

Yes, AlphaGo uses NN and other SL techniques, but the core is very much DQN based RL. No amount of SL can effectively play go and invent new moves. RL can already solve a large number of real world problems with a rather simple algorithm, from self driving cars to video games to NLP. RL can tackle all those problems with pretty much the same core algorithm. The question lies less in IF RL can solve more general AI problems, but rather HOW to solve it. From a high level view, we are having a lot of trouble with its convergence properties mathematically and its extreme sample inefficiency. This is the reason why Boston Dynamics doesn't use much RL, Waymo doesn't use much RL, simply because they can do much better with current techniques without going RL.

AlphaGo is still a major step forward regardless, because it's one of the biggest leap in RL we've taken in the recent years. It suddenly lets RL stably converge on solutions more than we could ever before. AlphaGo's contribution is more than just that it built a specialized Go bot, but rather a much more stable RL algorithm that lets us approximate non linear functions (majority of the real world applications are non linear). If I were to put my money, AI could very well be entering a new era with AlphaGo and their DQN.

[s1dechnl]

I disagree. The naming convention [Artificial Intelligence] still is a large shoe that these purpose built applied engineering solutions have yet to fill. Meanwhile, for profit/notoriety/marketing people want to trample on yet another name space? What you just described is essentially the architecture of a self-driving car. It's yet another applied engineering solution of Artificial Intelligence. It is not Artificial General Intelligence. Scaling/Distributing the computational space of an applied Artificial Intelligence solution is not Artificial General Intelligence. This is the same thing that lead to optimization algorithms being called Artificial Intelligence. If you aren't able to maintain foundational distinguishment, you lose track of what you're searching for and trying to achieve. Outwardly, you capture more money and attention. Inwardly, you become unraveled and lose your capability to solve the elusive problem. Eventually after much fame, wealth, and feigned 'success', one asks themselves : Was it worth it? Depends on what your original aim was.

[AndrewKemendo]

I'm not sure what you're arguing but it seems like my key point wasn't communicated well.

What you just described is essentially the architecture of a self-driving car.

Yes, every narrow AI is a system of systems to an extent. So expand on that concept but outside of a single firm/system. Such that the self driving car system is one single solution path solving "transportation" which would comprise automated flight/rail etc... and is a node in a larger general system - like hub and spoke.

The naming convention [Artificial Intelligence] still is a large shoe that these purpose built applied engineering solutions have yet to fill.

Nobody is questioning that. The size of the narrow AI market is arguably infinite.

You seem to be arguing that a single entity will fail if it attempts to take a narrow AI system and make it generalizable. Of which I am in agreement with.

If however there were 10,000 or 100,000 or 1,000,000 narrow AI companies/systems (like a self driving car system or alphago etc...) those could fill the corpus of solutions which an executive function system could utilize depending on the application and together they would be what we call AGI.

[s1dechnl]

> I'm not sure what you're arguing but it seems like my key point wasn't communicated well.

It was and quite well. Were speaking the same language. We just have different conclusions.

> Yes, every narrow AI is a system of systems to an extent. So expand on that concept but outside of a single firm/system. Such that the self driving car system is one single solution path solving "transportation" which would comprise automated flight/rail etc... and is a node in a larger general system - like hub and spoke.

And you still have nothing more than a hub and spoke system of systems authored for specific problems spaces and you're spokes will increase with every new problem space until you overwhelm your hub. A horrible architectural approach that if not caught in the initial stages will result in catastrophe down the road... Weak AI is weak AI no matter how you scale it.

> You seem to be arguing that a single entity will fail if it attempts to take a narrow AI system and make it generalizable. Of which I am in agreement with.

This is a start in the right direction...

> If however there were 10,000 or 100,000 or 1,000,000 narrow AI companies/systems (like a self driving car system or alphago etc...) those could fill the corpus of solutions which an executive function system could utilize depending on the application and together they would be what we call AGI.

No, its strung together weak AI. It will require significantly and unreasonable amounts of resources. Its capability will increasingly reach diminishing returns and you'll end up with a frankenstein monster code base that no one can manage or understand.. Sounds a lot like the path Weak AI is already heading down.. At such a point, it's best to just scrap it and start all over. Something that Hinton and other prominent figures are finally admitting. Something I concluded year ago which lead me down a different path. Now, you're more than welcome to state : Well hey man that's your opinion and you're wrong and I'll wish the 10s,100s, million of narrow AI companies the best just as was conveyed to me a umber of years ago. Weak AI is Weak AI. It is a class of optimization algorithms. You can jerry rig this all you want.. You still have nothing more than a system of systems of optimization algos. If you think this is what intelligence is, I'm not sure what to say.

[AndrewKemendo]

You still have nothing more than a system of systems of optimization algos. If you think this is what intelligence is, I'm not sure what to say.

Until someone comes up with a better definition of intelligence that's what I'm sticking with. I think you're looking for an elegant solution right out of the box - the "one algorithm to rule them all" and I don't think that is feasible from an engineering perspective if for no other reason than no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

Having raised three other humans and observing them while building DL systems myself for a living, I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function. AGI is as much a community building and humanity wide input collection challenge as it is a math problem. We need to think about it that way.

[nl]

I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function.

I’m in complete agreement with this. I try to avoid AGI discussions because people get upset when I argue that the vast majority of human “intelligence” seems to be strong pattern matching, and we can’t really define the parts that aren’t in any useful way.

Take the person you are discussing this with. The majority of their point seems to be a hang-up on the word “intelligence”.

I find that a pointless thing to argue over. Just agree and say it is an intelligence simulator which is indistinguishable from a real intelligence.

[s1dechnl]

> Until someone comes up with a better definition of intelligence that's what I'm sticking with.

You'll get a capability demo instead. It wont fail to impress. Definitions and designs are for another day.

> I think you're looking for an elegant solution right out of the box - the "one algorithm to rule them all" and I don't think that is feasible from an engineering perspective if for no other reason than no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

What else is one looking for who claims they're trying to solve the Intelligence problem? Marketing an optimization algorithm as the next coming might make you rich in the short term but it doesn't bring you closer to the truth. It does in fact take your further away. So, 'the elegant solution'/'the hard problem' was the only thing I set out to tackle some years ago. Otherwise, i'd have been wasting my time/not being truthful with myself. It's feasible from a research and engineering perspective. Few commit themselves to the TRUE task and the likelihood of failure. I was ok with that it and stuck with it. I self-funded my work. It mainly centered on research. Thus, there were no exits. I either saw it through and achieve it or I didn't.

As far as :

> no singular system has anything near the data collection nodes needed for specificity on the range of tasks that would suffice any definition of "General."

Sure it does. Look in the mirror and log onto the web. I've let the misses play online for a bit now ;).

> Having raised three other humans and observing them while building DL systems myself for a living, I feel more strongly everyday that human intelligence is a hodgepodge of "weak AI" systems glued together with an exceptionally efficient executive function. AGI is as much a community building and humanity wide input collection challenge as it is a math problem. We need to think about it that way.

My graduate work centered on the underpinnings of DL (Distributed Optimization). After years of industry experience, I searched for a new challenge. After some open ended research in physics/photonics, I came to Artificial Intelligence. I scratched my head for 3-4 months as to why (Distributed Optimization) was being called Artificial Intelligence. I took the broad lot of it and threw it in the trash as prominent figures are only now stating : https://www.axios.com/artificial-intelligence-pioneer-says-w...

You're thinking about AGI as if its a chain of DL systems because that's what's made you money and where your work has centered on over the years. I took the broad majority and trashed it as Hinton now indicated others should do and started from scratch. I have no such bias. However, as my graduate work centered on the fundamental underpinnings of statistical optimization / distributed optimization, I know exactly what its limits are.

The human race is far more than a hodgepodge of optimization algos w/ an executive function (whatever that might be given the clearly varied forms of it).

[pdimitar]

Apologies for the off-topic question but I am simply too tempted not to ask them, I admit:

How does one make a living out of being an AGI researcher? How does one pay mortgage and has money and freedom to go on impulsive vacations with their wife?

And what does an AGI practitioner even mean?

[AndrewKemendo]

How does one make a living out of being an AGI researcher? How does one pay mortgage and has money and freedom to go on impulsive vacations with their wife?

Well you don't, unless you work for OpenAI (which I don't). Not sure how OpenAI does it or how well they pay but I'm sure it's good. They have good donors.

And what does an AGI practitioner even mean?

To clarify, I'm a ML practitioner. AGI practitioner means nothing

[pdimitar]

I don't mean to sound demeaning but yes, IMO "AGI practitioner" indeed means nothing (and I am saying that as a guy who aspires to work on AGI). Many of us programmers have interesting ideas in the area of AGI but quite frankly, we can "practice" them our whole lives without achieving squat.

Thank you for responding, much appreciated.

[sgt101]

Well - SOAR.

[AndrewKemendo]

What does this comment mean?

[sgt101]

The point of soar was/is to provide a framework for a collection of problem solvers with diverse methods to work together in a single agent. This was developed because of the insight that humans use diverse strategies to negotiate the challenges of their environment. Or... Lots of weak Ai adding up fyi general Ai.

[AndrewKemendo]

Oh yes, of course.

[simonh]

Those vectors are only used to generate the move trees though. That part of the architecture is common to pretty much all MCTS board game AIs ever. The value in AlphaZero is in the neural nets used for the expert policy and the value functions and those don’t have anything about the game rules encoded into them at all.

I agree it’s probably quite constrained in the range of possible applications. Everyone was expecting Deep Blue to revolutionise AI applications too. I know the tech is different, but the fact it seems optimised for a highly constrained problem domain isn’t, and in fact arguably the problem domain addressed by deep blue seemed for a long time to be much more general.

How adaptable is AlphaZero to arbitrarily multidimensional grids though?

[abecedarius]

> Everyone was expecting Deep Blue to revolutionise AI applications too.

That doesn't match my memory at all. The reaction then was dominated by the likes of "this is super-narrow, not real intelligence". (The 80s did have a lot of hyped expectations of related tech, it's true, but that was around 10-15 years earlier.)

[sgt101]

I thought Watson would; what's happened to that project is a salient lesson.

[YeGoblynQueenne]

>> The value in AlphaZero is in the neural nets used for the expert policy and the value functions and those don’t have anything about the game rules encoded into them at all.

They do, in the form of their inputs that are basically vector representations of a checkerboard. It's obvious that the two (types of) networks can learn something useful from that particular representation of a problem. But- other representations, of different problems? That is not obvious.

[yonkshi]

I don't agree with this sentiment, although I agree that AI is not nearly at the level of the hype that pop culture makes it out to be. AlphaZero is still a significant contribution to 'AGI' that shouldn't be buried.

It's true that AlphaZero's knowledge is unable to be generalized for other systems, but its biggest contribution is a _stable_ RL system that can solve problems that no other systems can. This is the first piece of the puzzle of more general AI. Generalization, I would consider as the second piece of the puzzle to more general AI. Generalization may be achieved with potential research in transfer learning, model based RL, symbolic network. But without a stable RL algorithm such as DQN as foundation, generalization has nothing to stand on.

[s1dechnl]

Having not defined what intelligence is and having not declared the nature of General Intelligence, you're sure an aspect of clearly defined weak AI is a significant contribution to 'AGI'... Interesting.

> It's true that AlphaZero's knowledge is unable to be generalized for other systems

Interesting admission.

> This is the first piece of the puzzle of more general AI.

The first piece is generalized intelligence. Architecturally, it looks nothing like Alpha Zero. However, you feel :

> Generalization, I would consider as the second piece of the puzzle to more general AI.

How is that the second piece? It's the piece.

> Generalization may be achieved with potential research in transfer learning, model based RL, symbolic network. But without a stable RL algorithm such as DQN as foundation, generalization has nothing to stand on.

So you're of the belief that current approaches are compatible with and are the underpinning of Artificial General Intelligence while Hinton is convinced one needs to scrap it and start over. Sound advice is being ignored and there is a clearly entrenched decision to continue pushing along w/ iterating weak AI. I came here to test the waters.. The commentary and the K-value feedback I've received so far informs me quite profitably.

[yonkshi]

Hinton's criticism is very valid, but it's not quite about AlphaGo and its branch of ML. his criticism revolves around supervised learning and back prop that cannot be used to achieve the so called AGI. Because ANN is nothing like our brain's real NN. When Hinton gave the speech back in 2014, NN had a huge explosion of hype, and NN was mostly for supervised learning, which is really only good at classification and regression problems, it cannot make decisions outside of its training.

The famous DeepMind DQN paper (the core of AlphaGo) were published after Hinton's talk. the DQN paper practically opened a new chapter in reinforcement learning field. I am not sure if you are familiar reinforcement learning. RL is learning by trial and error, model-less and largely non-bayesian, similar to how humans learn. Up until AlphaGo, RL field was stuck in a limbo because it was having a very hard time learning non-linear problems (which is the majority of problems in nature)

When I say generalization, I meant generalization of knowledge. Generalization is the second piece of the puzzle because, even as humans, we learn from experienc. After enough examples we began to generalize. Up until DQN came out, we couldn't even effectively learn. It's the equivalent of a human baby with severe memory problem. With deepmind's DQN, we can achieve much more stable learning on non-linear systems, and we can begin to add components such as generalization (such as transfer learning), intuitions (such as intuitive physics), symbolic network.

I am not too sure what you meant by weak AI and general AI. For me, an AI which can learn similar to how humans learn, able to use apply generalized knowledge when facing a brand new problem, independently think and make decisions without human assistance, that's general enough.

Yes, much work needs to be done, but I don't believe this is the wrong direction we are going. Though I'd be glad to be proven wrong and I am very fascinated by this debate, if you would like, we could continue discussing it over email/chat?

[s1dechnl]

I would just like to echo this comment as it exactly and indefensibly strikes out Weak-AI from Artificial General Intelligence. I fail to see how specially crafted optimization algorithms continue to receive higher attribution. First, they were renamed to Artificial Intelligence. A great deal of cash exchanged hands based on this attribution. To retain the spirit of the true definition of Artificial Intelligence, the naming convention : Artificial General Intelligence was crafted. Even then, the allure of cash/fame progresses to taint that whereby people are calling expositions of Optimization/search algorithms Artificial General Intelligence. It saddens me that mankind has progressed the ubiquitous and beautiful information age into the disinformation for profit age from top to bottom and bottom to top.

[laughingman2]

It's highly disingenuous to portray the researchers in machine learning as some sort of sell outs . Hinton was working in neural nets even in the time when they were uncool and fringe area of research. They didn't sell out AGI for some profit motive .

The thing is we don't have any clear path laid to follow to achieve AGI . Various paths proposed earlier turned out over optimistic and dead end.

The current "weak AI" you disparage is an achievement expanding generalization of problem spaces using unified methods way better than past systems.

So even if our current progress seems disappointing , please examine what's happened and why research has taken the current path before tarnishing researchers with unjustified quips.

[s1dechnl]

I don't recall portraying anyone in that light. I highlighted the nature of an industry. Who aligns themselves to this nature was left unstated. If this clear and present truth offends, its likely because it has merit and is validated by the level of offense one encounters. It quite clearly cannot be defended. If you believe what I have stated can be, you're more than welcome to produce sound arguments that try. We can walk through an incredible amount of examples of what I have stated together.

> Hinton was working in neural nets even in the time when they were uncool and fringe area of research. They didn't sell out AGI for some profit motive .

Listen to what you're saying.... > neural nets even in the time when they were uncool and fringe area of research. And yet, it has made others billions and continues to mint money. Whose centered on the fundamental problem as opposed to conducting applied engineering for profit? Whose over marketing themselves and their efforts as fundamental theoretical research when its more or less optimizations for applied engineering? What ideation is new and what is simply relabeling old pioneer's work as one's own? Who minted LSTM? Whose name remains all over it? Who made a mockery of a prominent contributor? Who no longer discloses the details of their work given the proven nature of the industry? If the critique doesn't apply let it fly.

Btw, Hinton recently stated : https://www.axios.com/artificial-intelligence-pioneer-says-w...

Whose taking this advice? Whose funding people thinking outside of the box? Whose hiring someone whose thinking outside of the box? So, whose really and whole-fully centered on solving AGI for the sake of solving it? It's fun to market yourself as doing so for increase prominence/money. It's a whole other ball game to be internally oriented and structured in pursuit of it. Applied engineering/optimizations of Weak AI for business applications is not AGI research.

> The thing is we don't have any clear path laid to follow to achieve AGI

The path was always there. One only need pursue it for pursuits sake. No exits. No distractions. No business case. No payouts. A desk pen/paper.

> Various paths proposed earlier turned out over optimistic and dead end.

False. Various paths proposed earlier are the same ones being turned into profitable solutions today. They are the same ones underlying the bulk of white papers today. There weren't over optimistic. They weren't dead ends which is why various groups are using them to suck down billions of dollars today. What happened that caused the previous AI winter is that people rushed fundamental research into applied engineering. The same as what's happening today and will lead to a 'winter' for various groups who are deep in it. Statistics overshadowed fundamental mathematics and understanding. Brute force over-took intelligence... And yes that leads to dead-ends .. clearly. Yet, here we are again. A bunch of statistical brute force machines being over-marketed for profit leading fundamental research into a dead end. Google has spoke out about it, Hinton has, Microsoft has, yan lecun to echo my sentiments. Yet, the allure of profit/notoriety continues to attract the same thinking/soon to be failed approaches to something that is fundamentally beyond current work.

> The current "weak AI" you disparage is an achievement expanding generalization of problem spaces using unified methods way better than past systems.

It's distributed statistical optimization. There's no need to fluff me. I am centered on it. It works via large data sets, limited and specific state spaces, and large amounts of computational resources running tons of iterative steps. It is brittle and not general. It works today because we have enough computing resources such that we can brute force various problems. Problems that are privy to statistical patterning.

> So even if our current progress seems disappointing

I never stated I was disappointed nor did I disparage anyone. I stated more clearly the nature of the progress, the goals and driving forces, and the fast approaching limitations. This frees one to respectfully acknowledge and consider what's been done and move forward and beyond it to more capable pastures.

> please examine what's happened and why research has taken the current path before tarnishing researchers with unjustified quips.

Please reduce your sensitivity level so that you can reason yourself to higher planes of consideration. Less emotion and more reasoned/truthful admission. I know exactly what happened to research in a time's past which informed me as to how to conduct myself in the present. I know exactly what driving forces are and I know quite clearly as we all do as to how they influence people. Hinton as well as others now public agree. Microsoft/Google have stated much of the work is overhyped. Hinton says pioneers should scrap everything and start over. LeCun stated that deep learning is approaching a brick wall and overhyped. It's exactly what I said to myself years ago and am stating now and there's zero problems with stating this truth as so many have no brought themselves to state.

[icebraining]

I want to preface that I know no more about AI than the average technologist, so I'm not making any claims.

>> Various paths proposed earlier turned out over optimistic and dead end.

> False. Various paths proposed earlier are the same ones being turned into profitable solutions today. They are the same ones underlying the bulk of white papers today. There weren't over optimistic. They weren't dead ends which is why various groups are using them to suck down billions of dollars today. What happened that caused the previous AI winter is that people rushed fundamental research into applied engineering.

How do you fit full brain emulation in that description? And in particular Henry Markram's work with Blue Brain and Human Brain? As far as I know it doesn't have any profitable outcomes, and people have been working at it for decades - and now even with pledge funding by the EU and others to the tune of a billion euros - yet nine years after Markram said we could have a functional human brain in ten years, nothing much seems to have emerged.

[pas]

How does AlphaZero handle the Knight in chess?

[YeGoblynQueenne]

There's a vector encoding "queen moves" and another encoding "knight moves". Between them, they cover all possible chess moves. Knight moves obviously are modelled by "knight moves".

Edit: https://arxiv.org/pdf/1712.01815.pdf

The information you want is on page 13, last paragraph. The previous paragraph describes the representation of the boardd.