[karmasimida]

And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants).

Such architecture works great for differentiable data, such's images/audios, but the improvement on natural language tasks are only incremental.

I was thinking maybe DeepMind's RL+DL is the way leads to AGI, since it does offer an elegant and complete framework. But seems like even DeepMind had trouble to get it working to more realistic scenarios, so maybe our modelling of intelligence is still hopelessly romantic.

[sdenton4]

Maaaaybe. I tend to think that symbolic reasoning is a learning tool, rather than a goalpost for general intelligence. For example, we use symbolic reasoning quite extensively when learning to read a new language, but once fluent can rely on something closer to raw processing - no more reading and sounding out character sequences. Similarly with chess - eventually we have good mnemonics for what make good plays, and can play blitz reasonably well.

And - let's be real - a lot of human symbolic reasoning actually happens outside of the brain, on paper or computer screens. We painstakingly learn relatively simple transformations and feedback loops for manipulating this external memory, and then bootstrap it into short-term reaction via lots of practice.

I tend to think that the problems are: a) Tightly defined / domain-specific loss functions. If all I ever do is ask you to identify pictures of bananas, you'll never get around to writing the great american novel. And we don't know how to train the kinds of adaptive or free form loss functions that would get us away from these domain-specific losses.

b) Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. We currently mostly build models that are only receptive (image, sound) or generative. Reinforcement learning is getting progress on feedback loops, but I have the sense that there's still a long way to go.

c) I have the feeling that there's still a long way to go in understanding how to deal with time...

d) As great as LSTMs are, there still seems to be some shortcoming in how to incorporate memory into networks. LSTMs seem to give a decent approximation of short-term memory, but still seems far from great. This might be the key to symbolic reasoning, though.

Writing all that down, I gotta say I agree fundamentally with the DeepMind research priorities on reinforcement learning and multi-modal models.

[joe_the_user]

Once someone is fluent in a language, the logical operations and judgements involved stop being overt and highly visible to the conscious mind. But that doesn't mean that one stops getting the benefits and results of logical operations.

What you might see as logical operations "not mattering", I would see as logical operations integrated so deeply into reflexive operations that it's hard to see where one ends and the other begins. The contrast is that humans can do pattern recognition in a neural net fashion, taking something like the multidimensional average of a set of things. But a human can also receive a language-level input that some characteristic is or isn't important for recognizing a given thing and incorporate that input into their broad-average concepts. That kind of thing can't be done by deep learning currently - well, not a non-kludgey sort of way.

Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs.

It depends on how you want to mean that. A human can take inputs on one thing and apply them seamlessly to another thing. Neural nets tend to be very dependent on the task-focused content fed them.

[visarga]

I think "a better set of inputs" is the real world or much better simulators to train our RL agents. François Chollet (author of Keras) was saying a similar thing - focusing too much on architectures and algorithms we forget the importance of the environment, an agent can only become as smart as the hardest problem it has to solve in its environment, and depends on the richness of said environment for learning. Humans are not general intelligences either, we're just good at being human (surviving in our environment). We'd be much smarter in a richer environment, too.

https://intelligence.org/2017/12/06/chollet/

[barrkel]

There's a parallel between something being logical and it "feeling right" without a necessary connection at the "implementation level" between the two, just like there may be a parallel between an artificial NN recognizer recognizing something unambiguously and not caught awkwardly with multiple weak or conflicting activations, and a logical system using rules to determine a contradiction, without ever needing to embed the second in the first, however deep - it's just that illogical inputs didn't get good training because they either don't happen or have no meaningful training data.

I, personally, just know I don't use logical rules very often at all. Usually I apply them retroactively as a post-hoc justification, or narrative, to explain a sense of discomfort or internal conflict or dissonance, but I have no way of knowing if my rationale is true other than how it makes me feel - I'm simply relying on the same mechanism, with an extra set of pattern recognition learned specifically to identify fallacies and incorrect logical constructs. If I didn't have that extra training, my explanations could be illogical and I'd be none the wiser.

I think humans are very bad at logical reasoning and very inefficient at it. Only a small % of the population ever does it and they usually do it incorrectly with biases, constructively to justify an already held conclusion. They're great at pattern recognition though. I don't think logical reasoning is anywhere on the critical path to human level AGI at a deep level. It could very well be a parallel system though to help train recognition if we don't figure out better ways of doing that.

[joe_the_user]

Well, neural nets and similar things laughably worse than AI systems when confronted with "real world" situation.

I wouldn't argue with the point that humans use rigorous logic and overt rules-based behavior much less than they imagine (your summary is very much a summary of the other-NLP model of mind, which I know).

I'd argue that while "refined" logic, systematic logic, might be rare, fairly crude logic, more or less indistinguishable from simply using language, is everywhere and it an incredibly powerful tool that human have. Again, being able to correct object recognition based on things people tell you is an incredibly powerful thing. You don't need a lot of full rationality for this but it gets you a lot. And that's just a small-ish example.

[Retric]

Intelegence is not limited to what Humans are good at. People are really bad at several tasks where current AI tech excels, but those things tend to be excluded from the conversation.

AGI that is as smart as say a rat would easily qualify as AGI even without language skills.

[joe_the_user]

Intelligence is not limited to what Humans are good at.

Being able to implement all the things human are good at, however, should be able to get us everything that we could do, because anything we could create, it could create too.

AGI that is as smart as say a rat would easily qualify as AGI even without language skills.

Indeed, but while a full language-using AI is ways a way at least, using language is one thing that's at sort-of describable/comprehensible as a goal. A rat is a lot more robust than any human made robot but how? Overall, I keep hearing these "there's intelligence that's totally unlike what we conceive" argument but it seems like computer programs as they exist now either do what a human could do rationally and more quickly (a conventional program) or heuristic duplicate human surface behavior (neural nets). You could sort-of argue for more but it's a bit tenuous. Human behavior is very flexible already (that's the point, right). And assuming AI is hard to create, creating something who properties we to-some-extent understand is more like than creating the wild unknown AI.

Also, "Getting to rat level" might not be the useful path to AGI. If we simply created a rat like thing, we might win the prize of "real AGI" but it would be far less useful than something we could tell what to do the way we tell humans what to do.

[mindgam3]

"Similarly with chess - eventually we have good mnemonics for what make good plays, and can play blitz reasonably well."

"let's be real - a lot of human symbolic reasoning actually happens outside of the brain"

I was a chess master at age 10. Let's be real - when I play blitz and bullet chess, I am performing multi-level symbolic reasoning at multiple frames per second. In my brain.

I am not an alien. I can do these kinds of symbolic calculations faster than 99.6% of the population mainly because I learned chess as a kid, making it a "native language", and I got good at it early so I spent much of my youth training my neurons with this perceptual task.

My point is not to claim I'm a genius. There are dozens of players who can school me in bullet the way I can school most people.

My point is that human beings DO symbolic reasoning, it is the core of our intelligence. Being able to take in different kinds of input, organize some of them into relevant higher level clusters, sort the clusters by priority, make a plan to deal with the highest prio clusters, act, rinse and repeat.

Humans simply do not have the computational ability to make decisions based on raw perceptual data in real time. Our brains are designed to act on higher levels of symbolic meaning, and we have perceptual layers to help us turn reality into manageable chunks.

In cognitive psychology this is referred to, not surprisingly, as "chunking": https://en.wikipedia.org/wiki/Chunking_(psychology)

Until DeepMind starts working on anything resembling chunking, I believe they are wasting their time and money.

[meroes]

As a layman, is this just saying we learn by training an intuition of what's what/what's correct, rather than actually calculating deep reality/referencing our entire memory set every time we intake some information or need to solve a task problem? Meaning, we develop tons of rules/heuristics after repeated pattern exposures, and use the simplified rules rather than a deep theory or 'brute-forcing' every possibility until we find one that's right. For example with chess, we don't know at the deepest possible level why this move might be best, it just feels right due to a massive learned intuition.

If that's the case, then to me it seems like AGI is limited by the amount and type of data a NN can be fed. To have an intelligence like homo sapiens, wouldnt you expect that no matter the underlying NN, it has to take in a comparable amount of data to what the 5+ human senses take in over lifetime, plus the actual internal 'learning' (i.e pattern recognition, heuristics, and intuition) + some kind of meta awareness (consciousness) to speed up and aid this process + dedicated pieces of the brain such as Broca's/Wernicke's

[TheOtherHobbes]

AI is a confused soup of more or less (un)related concepts: agency, sentience, pattern recognition, unsupervised learning, embodiment, NLP, and goal selection - among others.

IMO the minimal useful definition of AGI would list a set of testable skills that would qualify as AGI, and a more useful definition would be based on quantifiable skill sets that would allow numerical comparisons between humans and AIs.

It seems pointless to speculate when AGI might be a reality when we have only the fuzziest idea what AGI is supposed to look like.

[karmasimida]

I put symbolic reasoning at the spotlight for it is something that NN is particularly bad at: discrete data, hard to design, often approximate and non-differentiable measurement.

The problem is so inherently hard that we are struggling even to come up with a meaningful task, telling us how bad we are doing. That comes to your first point, I think finding the right loss function is a like a chicken-and-egg situation here. When you have the loss function at hand, you already what task and problem you are going to solve, then it becomes easier. But that is apparently not our current situation.

That is why I think DeepMind has a good reason to go after reinforcement learning, after all, that is how we human are trained, through exams and the feedbacks.

As to your point about LSTM, I am not very passionate to qualitatively claim it whether it can/can't handle short/long term memory. That is apparently task dependent, and all the concepts involved are ill-defined.

[kahoon]

I don't understand this fixation on symbolic reasoning. Do any other animals practice this? If the answer is no, then it is probably not the most important milestone to AGI or at least not the one we should be currently aiming for. Right now we can not replicate the cognition of a mouse. Feels like we want to go to Mars before figuring out how to build a rocket.

[evrydayhustling]

Seconded. Even if animals do symbolic reasoning, they do it on top of hardware based on continuous physical dynamics, more similar to DNNs... So why not build on that platform?

I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence... But I don't see where there is evidence that symbolic reasoning is either necessary or sufficient for AGI, except people describing how they think their brain works.

Related, there are a lot of statements that symbolic or rule based systems do better / as well as / almost as well as neural methods. Citation please, I'd love a map of which ML problems are still best solved with symbolic systems. (Sincerely - it's not that I expect there aren't any.)

[kahoon]

> I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence...

Good point, we wouldn't have AlphaZero now if we only relied on biological inspiration. Nature hardly ever performs Monte Carlo Tree Search (though I'm not sure this is entirely true, see slime mold searching for food: https://thumbs.gfycat.com/IdealisticThirdCalf-size_restricte...).

[visarga]

We're also good at trying out various ideas until one sticks. Isn't that MC tree search?

[taneq]

The thing is, whatever the hell it is that human brains actually do in the background to produce our 'understanding' of the world and our ability to synthesize new ways to manipulate it, we're also very good at back-fitting explanations based on symbolic reasoning. So it looks like machines need symbolic reasoning to replicate human abilities, whereas I'd bet a dollar that actually, we're doing something quite different (and messy and Bayesian and statistical) in the background and then, using the same process, coming up with a story to explain our outcome semantically. It's not insight so much as parallel construction.

[barrkel]

I fully agree, as I wrote in my other comment in here. Logical symbolic reasoning is usually post-hoc rationalisation built constructively to come to an already held conclusion that "feels right". It's rare that someone changes their mind due to logic, especially if the topic isn't abstract and has real-world consequences and emotional engagement.

[visarga]

> usually post hoc rationalisation built constructively to come to an already held conclusion that "feels right"

Counterfactual reasoning is a promising direction for AI. What would have happened if the situation were slightly different? That means we have a 'world model' in our head and can try our ideas out 'in simulation' before applying them in reality. That's why a human driver doesn't need to crash 1000 times before learning to drive, unlike RL agents. This post hoc rationalisation is our way of grounding intuition to logical models of the world, it's model based RL.

[Gibbon1]

I think the the fixation on symbolic reasoning comes from ignorance at how hard classification is vs how hard pure mechanical symbolic operations are for humans. It's easy to make the mistake of thinking that since a computer can rapidly multiply two numbers together (hard for humans) that they were operating at a higher level than human brains.

Turns out this is wrong. Human brains are very efficient.

[mehh]

> Human brains are very efficient.

At some things, not all.

Subsymbolic systems, such as ANN are clearly good at some things and symbolic systems are better at others.

It is argued that symbolic reasoning is required for what we might call higher levels of intelligence (lets assume this is correct).

Symbolic systems have struggled in the realms of grounding a symbol to something in the physical world, because its messy and complex, i.e. the area where subsymbolic systems play best.

If we assume that ANN are approximately akin to natural brains, then can we take that they are examples of a subsymbolic system able to, with the correct architecture, produce (perhaps the wrong word) a symbolic resoning system?

Perhaps this emergence ontop of the subsymbolic processing is what humans (and others to varying degrees) possess. Perhaps in the past (GOFAI) suffered because it was going top down, or not even going down to subsymbolic to ground the symbols.

Perhaps ANN struggles because its not going up to symbolic reasoning.

Then also perhaps ANN (or organic brains), which evolved where reaction/perception give the critical survival advantage, then only much later did symbolic become possible and beneficial, however wit hardware that wasnt necessarily developed for that in most efficient way.

Being of the belief that ANN are sufficient for AGI (for 20+ years), and possibly offer an elegant solution, I currently think that they are at this time, not the most efficient (nor plausible with the current compute/hardware, or for many years (probably my lifetime)). Practical progress imho is likely in hybridisation of ANN and Logic (however I'm not referring to hand baked rules), and even propose a mixed hardware might even supersede a pure ANN or what evolution has provided in the brain.

[mindgam3]

100% agree. I am terrible at mental arithmetic, but I am exceedingly good at performing symbolic operations playing bullet chess. It's primarily a visual or geometric calculation, not purely abstract like math.

I think most people don't realize that our brains have this ability. But all you need to do is spend a few months learning chess and you'll see for yourself.

[darawk]

> And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants).

You think symbolic reasoning is not a function? In what sense do you think 'symbolic reasoning' is a distinct thing from 'function approximation'?

[Symmetry]

Something very interesting to me about the work Deep Mind has been doing is the way they've been combining neural network intuitions with tree search reasoning in Go, Chess, protein folding, etc.

[skybrian]

Hmm, it seems like natural language translation has been getting quite a bit better with statistical techniques, though? I guess it depends what you mean by "only incremental".

[tanilama]

I still remember sometime back Google's GNMT translate the Chinese text of 'I don't want to go to work' into English 'I want to go to work'. That example alone should be sufficient to showcase how most advanced machine learning model can fail at the simplest task.

It didn't understand the source material, it is just very good at memorizing and faking.

[joe_the_user]

Whenever I attempt to use a translate site to translate more than a paragraph (Facebook or Google), it comes out a garbled mess - doesn't mean some sentences are seemingly clear and meaningful. The big thing is it chokes on idioms, not understanding, not leaving them as is but guessing some clearly wrong meaning. I occasionally find single-sentence posts on Facebook apparently translated astonishingly well, in the sense of being literate English and seeming to reflect the original meaning. But my French or Spanish is quite rough so translate could have missed something big - as I know it does when you get into longer texts.

[narag]

Cada vez que intento usar un sitio de traducción para traducir más de un párrafo (Facebook o Google), sale un lío confuso - no significa que algunas oraciones sean aparentemente claras y significativas. Lo importante es que se atraganta con los modismos, no entendiéndolos, no dejándolos como están, sino adivinando algún significado claramente erróneo. Ocasionalmente encuentro que las publicaciones de una sola oración en Facebook aparentemente traducen sorprendentemente bien, en el sentido de que son en inglés y parecen reflejar el significado original. Pero mi francés o español es bastante duro, así que traducir podría haber pasado por alto algo grande, como sé que sucede cuando se trata de textos más largos.

I'm a native and it seems almost good to me. "algunas oraciones no sean", OK. And "duro" should be "rudimentario", also "de qué son" lacks the accent. But the rest is acceptable and it's possible to get a decent translation, only modifying those bits.

http://www.DeepL.com/Translator

[joe_the_user]

Oh, that just means I happen to write in clear, unidiomatic English ;-). Add just a smidgen of irregular usage, contemporary metaphors and such and things can go South pretty quickly.

[barrkel]

That'll just be a training problem (often translation is driven from example texts that have lots of translations, and we have many of these for multi-national orgs like EU, which necessarily don't include a lot of colloquialisms), and the inconsistency isn't found because the models built don't extend all the way out to real world experiences (training) and recognizing text as a real world experience narrative. I think a much deeper network could do better but we don't know how to train them, we take way too long as it is.

[narag]

I've seen the evolution of translators and there is a big difference in results. We can try more convoluted examples.

Also do you think that every human can parse that contemporary metaphors better?

[harperlee]

Statistically, but it is a shallow translation, with no modelling of what is said. This works astonishly well for translation, but gives the false intuition that it is meaningful; whereas it is in fact orthogonal to advancing in systems that comprehend a text so as to be able to reason their way out of winograd problems.

[m0zg]

It's been "statistical" since about mid-00's. What's new is that it's now neural.

[cantthinkofone]

If humanlike reasoning is the destination for AGI, there's more than just symbolic reasoning to factor in. Emotions are a huge control on human reasoning.

People essentially rely on emotions to make all their decisions. Emotions implicitly represent rapid-fire unconscious decision work.

Again the current popular understanding of the mind separates emotion from thinking. They are not distinct. Emotional processing is another kind of thinking, and it drives the show.

[visarga]

I see emotions as analogous to the value function in RL. It is essentially a prediction of future rewards based on current state and action plan. Artificial RL agents learn emotion as it is related to their tasks and environments.

[odyssey7]

Are there ways that an AI practitioner would be able to tell whether a neural network is doing human-style symbolic reasoning?

[bsaul]

As mentioned in the original article, being able to reuse part of a network trained on one task, on another different task that shares a subset of concepts, would indicate something like the understanding of a concept has emerged.

[karmasimida]

Good question. I don't think we do actually.

The only reason I am convinced it is NOT doing a good job, is how utterly difficult to apply NN to dialog generation/management domain of business, often time it behaves much worse than rule-based systems.

[darkmighty]

You'd be able to tell if every brain structure was replicated by a NN analogue (and we understood them sufficiently well). Otherwise you can only use behavioral replication (i.e. Turing tests) to infer it.

[antidesitter]

IQ tests [1] and one-shot learning come to mind.

[1] https://arxiv.org/abs/1807.04225