[dreamcompiler]

Symbolic AI fell out of favor because it was overhyped. It was delivering quite impressive results--just not the promised results. Neural nets fell out of favor in the 90s for exactly the same reason.

Both failures ultimately were caused by not enough computing power. Even though Deep Learning and Convolutional NNs look like major advances today, they never could have been practical before about 2005: There just wasn't enough computing power.

If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains.

[chillacy]

I don’t have enough knowledge to evaluate the idea that symbolic AI would scale as well as deep learning (or could I say statistical models in general?), but what has held back symbolic AI if not for the difficulty of making it work? Surely someone out there has tried in the past 5 years if the reward is that great.

[dreamcompiler]

It's mostly because symbolic AI is not fashionable. Most AI practitioners under 40 (and VCs) seem to think AI is DL and only DL.

[jedharris]

Google, Apple, etc. tried to make symbolic AI work for their problems but found it had intractable problems. Google fairly publicly transitioned to a deep learning approach for translation which greatly improved their results. Apple is still struggling to make the transition for Siri's question answering.

[chillacy]

How might symbolic AI become fashionable?

[hodgesrm]

> If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains.

What's the basis for this conjecture? Is there a mathematical model for symbolic manipulation that would benefit from parallel execution/GPUs the way ML applications do?

[YeGoblynQueenne]

Symbolic AI needs computing power to counteract combinatorial explosion.

https://en.wikipedia.org/wiki/Combinatorial_explosion

The vulnerability of early logic-based AI to combinatorial explosion was the main argument against funding AI research put forward in the Lighthill Report, the document that shut down AI research in the UK in the 1970s and contributed to the AI winter on the other side of the Atlantic, also.

Obviously, today we have more powerful computers so combinatorial explosion is less of an issue, or anyway it's possible to go a bit further and do a bit more than it was in the '70s.

One area of symbolic AI that actually does benefit from parallel architectures (though not GPUs) is logic programming with Prolog. Prolog's execution model is basically a depth-first search, which lends itself naturally to parallelisation (one branch per search). Even more so given that data in Prolog is immutable (no mutable state, no concurrency headaches).

But, in general, anything people did 20 or 30 years ago with comptuers can be done better today. Not just symbolic AI or neural networks. I mean, even office work like printing a document is faster today and that doesn't even depend on GPUs and parallel processors.

[YeGoblynQueenne]

>> (one branch per search)

Oops, sorry. Meant "one branch per processor".

[dreamcompiler]

My conjecture is about e.g. the Rete algorithm for rule search and the likelihood that it could be made more scalable on multicore and distributed hardware with modern functional data structures allowing easy rule updates.

I don't know whether rule search or logic unification could be mapped onto GPU or TPU operations; I suspect not, but it's worth looking into.

[jeremyjh]

You realize someone has to write all those rules - and correctly - right? Are you saying that we literally didn't have the computing power to run all the rules we could actually write? I think you need to support this idea that symbolic approaches failed to lack of computing power.

[YeGoblynQueenne]

Like I say in another comment, machine learning took off in part as a way to avoid having to hand-craft rules for expert systems' rule bases (although machine learning existed as a discipline from the early days of AI). So a lot of work on machine learning in the '80s and '90s went to learning rules.

For instance (also in another comment) Decision Tree learners basically learn a set of If-Then-Else rules. They're one type of symbolic machine learning and there's more where they came from (e.g. Ross Quinlan's FOIL, for First-Order Inductive Learner, which is basically a first-order version of decision trees; Inductive Logic Programming which I study for my PhD; and many, many more). This work has dwindled, but it's still going.

So, no, you don't have to write rules by hand, anymore than you need to set the weights of a neural net by hand. You can just learn them.

[malmsteen]

Yeah so why not just do it yourself ?

I really hate these kind of arguments where some people have strong beliefs that "it would work" and not do it or invest themselves. Its like being in a pub and listening people talk about politics while neverhave had any responsibilities

[jedharris]

There are plenty of domains where symbolic AI has had every opportunity to use unlimited computing power, plus decades of design and experimentation, but has lost to deep learning.

a good area for examples is human games (e.g. chess, Go, Atari games, etc.) Symbolic AI has been pushed hard but has lost definitively to deep learning. Furthermore symbolic approaches had decades of investment, compared with less than a decade for the deep learning approaches.

Another good area for examples is natural language. Marcus admits that deep learning is the only viable approach to "speech understanding" (which really means transcription). He doesn't mention translation which demands a lot more "understanding" and where deep learning excels relative to symbolic approaches, again with decades of investment on the symbolic side and much less on the deep learning side.

Except for inherently symbolic problems like theorem proving, I can't think of any AI domain where deep learning doesn't dominate or seem likely to dominate symbolic approaches.