[LionessLover]

Depends on what you mean by "predict". Also, unfortunately I don't even remember where I saw or heard this in a lecture, but when studying neuroscience I remember to have seen exactly this question and an example showing that you actually don't have to make predictions. I also forgot the explanation that showed how the (real, biological) neural network solves just such a problem without having to make a prediction. I only have the fuzziest memory of it being a process and at no point was there any prediction of the path of the object being tracked. It was just matching several sensory signal inputs and creating outputs, something clever, using an indirect approach. "Predicting" would be observing the object for x amount of time, doing a calculation where it will be some time later, using a model to come up with a way to intercept, then creating outputs, all of that in a loop, something like that. In any case, the way the neural network actually solved it was completely different from how an engineer would do it. In a sense, the neural network was "cheating" and doing far less work than you would expect.

The one thing I do remember for sure was there was no "prediction" involved - none at all. Unless you argue backwards and say because it succeeded you declare the process a "prediction". Once explained the whole process was actually quite primitive. Again, that was research on an actual biological neural network.

Darn, now I wish I had paid more attention. Any actual neuroscientists here? Without the details even I myself can't see my own comment as a satisfactory reply, but only as a step to actually getting one from somewhere or someone else. But note that it depends on what you mean by "prediction" - as I said, if you define it backwards from success than sure, prediction happened. My point is that the process is very different from how a human-made algorithm would do it.

[jamesrcole]

Think of trying to catch a fast moving object. Neutral process are relatively slow. Then they need to send nerve signals to muscles in the arm and hand. Then the muscles have to contact. All these things take time.

That means the brain processing had to be done in anticipation of where the ball will be. That means it has to predict where the ball will be.

I have a feeling that you're using an overly narrow meaning for "predict"

[amatic]

There are different ways of predicting. There is a series of research on catching fly-balls that claims there is no prediction in the sense of "estimate/predict end position, move to estimated position"; instead there is an ongoing process of "maintaining the visually perceived velocity of the ball". You end up catching the ball just the same.

http://www.mindreadings.com/OpticalTrajRM.pdf

http://www.mindreadings.com/ControlDemo/CatchXY.html

[jamesrcole]

I mean 'predicting' in the general sense of needing to arrange the details in advance, which is a necessary requirement on any real-time system.

.

EDIT: to all the responses objecting to what I'm saying

- I'm aware of the details you are talking about, and I believe you are reading much more into what I'm saying than I have actually said.

- do you deny that the brain has to set in motion the muscular activities in advance?

- you seem to all be working on a very narrow notion of what predicting means - that it must be some explicit calculation of coordinates.

[amatic]

Then that is probably not the way human or animal brains do object or pray or something catching. First, you would need very accurate estimates of current velocity and direction of the object, but human sensory systems are fairly imprecise. Still, if the details are calculated in advance you would expect that, say, a baseball catcher would be able to predict exactly where the ball would fall after the initial visual estimate. This does not happen: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3816735/

> even skilled baseball players can’t identify correct ball trajectories or predict landing points

[jamesrcole]

to take one simple example to illustrate the point:

light comes from a moving object

by the time the light has reached the perceiver the object has already moved (in most cases, very slightly) forwards.

by the time the brain has done some bit of processing to do with that object, the object has moved further. brain processing is not very fast.

say this processing generates nerve signals. by the time those signals have reached muscles, the object has moved even further.

by the time those muscles have contracted the object has moved further still.

what i am arguing against is the assumption that all these details are instantaneous. they are not. this is a fundamental constraint.

that means that whatever that brain processing is doing, and however the nerve signals are "telling" the muscles to do, while it is based upon where the object was at one point in time, fundamentally must concern where it is some moments later on.

if you don't want to call this a form or predicting, then fine, though I would like to know why you think the term 'predicting' does not suit what is going on there, and I'd like to know what alternative term you would use to describe what the brain processes "concern".

[amatic]

At low levels, I think it is probably a very simple computational process, something like taking a derivative of the position (estimating the direction and magnitude of velocity of the object in some perceptual space). If we understand prediction in a loose sense, then taking the derivative would be predicting, just like the D part in a PID loop is considered predicting.

At higher levels of the nervous system where more abstract variables are hypothetically handled, there could be something like model-based prediction. For example, if we are trying to catch a ball we would predict its movement differently then if we are trying to catch a cat or mouse or a drop of water.

[joncrocks]

I think what the parent is trying to say is that there's more than one way to ensure a given event occurs.

Imagine that I have a specialised piece of brain that deals with catching a ball in flight. That piece of the brain fires on the current input and makes adjustments to the position of my hand appropriately.

In the micro sense, in terms of the feedback loop involved, my brain might be predicting the path of the ball for the next tens of/hundreds of milliseconds. But there's no way to ask it "where is the ball going to be in 3 seconds time", only "what should my hand do next."

So in the macro sense, the brain isn't predicting the path of the ball, only what should be done in any given instant.

Or something like that....

[jamesrcole]

> the brain isn't predicting the path of the ball, only what should be done in any given instant.

"what should be done in any given instant" is a fantasy. Brain processing takes time, nerve signals take time to travel, muscles take time to contract.

Nothing is instantaneous. It all takes time.

It's actually "what should be done a number of moments in the future". And that is a kind of prediction.

I must admit I'm finding all this discussion pretty frustrating - this is really elementary stuff!

[Noseshine]

> do you deny that the brain has to set in motion the muscular activities in advance?

What do you mean by "in advance"? What does that mean? That's an empty phrase. In advance to what? When it moves the arm to catch the ball it moves to what is there right now.

> you seem to all be working on a very narrow notion of what predicting means - that it must be some explicit calculation of coordinates.

And you seem to have a very lenient definition. The guy you responded to already said that if you define the final outcome as a "prediction" then it is one. But that is meaningless, such a loose usage does not gain you anything useful.

Everything you write, for example

> That means the brain processing had to be done in anticipation of where the ball will be

seems to be a gross misunderstanding of what's going on in the brain. But others have written enough already.

There is no such thing as "anticipation" unless you go to a meta-level of interpretation that is not useful to understand those processes. You simply declare success because it succeeds - but as others pointed out, the methods are so different that using the same terms is actually bad.

[jamesrcole]

> > do you deny that the brain has to set in motion the muscular activities in advance?

> What do you mean by "in advance"? What does that mean? That's an empty phrase. In advance to what? When it moves the arm to catch the ball it moves to what is there right now.

It mean it according to the standard meaning of that phrase. It's not empty.

Here's an example to illustrate my point. It's simplified to make the point easier to understand.

The ball is moving.

At time T, the ball is at position X.

At time T the brain has received the visual information about the ball at X. It then processes this information. The amount of time this processing takes is A1.

It is now time T+A1. The ball is has moved on to position X2.

At time T+A1 the brain sends signals S to the muscles. This takes A2 amount of time to reach the muscles.

It is now time T+A1+A2. The ball is now in position X3.

The muscles do their contractions for reaching out and grabbing. This takes A3 amount of time to happen.

It is now time T+A1+A2+A3. The ball is now in position X4.

The brain started this process when the ball was at position X, and had to determine the signals S to send to the muscles based on details of the ball being at position X. But when those signals actually play out the ball is at position X4.

.

What I'm talking about is extremely elementary - it is simply the fact that perceptual+cognitive processing, plus nerve signals, plus the actions of muscles all take time! What you and the others are arguing would essentially require the absurd notion that it all happens instantaneously.

[EDIT: fixed some wording]

[LionessLover]

> What I'm talking about is extremely elementary

Whenever someone doesn't have an argument they resort to such empty phrases, merely repeating over and over "I am right!". Maybe you should have studied some neuroscience LIKE I DID, than you would not be left without arguments in discussions about neuroscience.

> plus the actions of muscles all take time!

And yet there is no "prediction". As has been pointed out to you by a lot of people including myself repeatedly. Cognitive dissonance is strong in "jamesrcole".

[joe_the_user]

- you seem to all be working on a very narrow notion of what predicting means

Again, the original article is about a program "better than people" at predicting explicit coordinates. Whether the brain is predicting in some more abstract way is an interesting question but more semantic since such other ways aren't necessarily testable.

[jamesrcole]

You said:

> The thing is that I don't think what humans are good at is statically predicting the movement of objects from video.

> Rather, I think the strong human skill is reacting appropriately and quickly to movements."

I think that implies that "reacting appropraitely and quickly to movements" is different from predicting. That's what I was responding to.

[LionessLover]

But the point is that it does not do that. A neural network does not work like a computer. It does not have to predict. It is a parallel flow from input to output AT ONCE. There is no "processing" like in a CPU where it takes n amount of CPU cycles and then the result is sent on. And as I said, it uses a proxy - it does not try to predict anything, it uses the data it has at that moment and nothing else. Before you get mad at me, do take some neuroscience courses please. I'm an IT guy myself and it opened a completely new world for me. Arguing with someone who only sees one side is frustrating. And while I'm not good enough to be able to explain the neuroscience - maybe not at all, definitely not in a forum comment - I still know a little bit about the subject. "Prediction" and "Looking ahead" may be system outcomes, but it does not actually happen as part of the actual low-level process. Not for the low-level processes like catching a flying object, I'm not talking about conscious thought processes.

When a moving object leads to input from different retinal ganglion cells - always in the form of action potential frequencies (so, an analog signal despite an action potential being all-or-nothing, just an aside) through temporal summation timing differences - which can be a function of the speed the object is moving in the real world - can lead to different subsequent processing neurons being activated, eventually leading to different motor neurons being activated or the same ones firing at different rates. So the computation takes place with the signal flowing as a "wave" across brain regions, but it all takes place at once. There is no "let's calculate where this is going to be in a second". This is implicit by connecting input directly to output through paths that change in subtle ways depending on said input. Yes, the end result (system outcome) is a "prediction", but not in the same way as a computer would do it.

It just "happens", there is no actual effort to predict anything. There also is no representation of such a "prediction" anywhere else: It flows right into your movement, but as somebody else has already pointed out just because you manage to catch the ball doesn't mean you are any good at consciously being able to make actual predictions.

By the way, the processing already starts in the retina, which consists of several layers of cells, and the ganglion cells that communicate with the visual cortex at the very back of the head (after being relayed through the geniculate nucleus of the thalamus in the middle of the head). They don't provide a signal like a camera CPU gets from an RGB chip which simply 1:1 sends pixel values. You have cells signaling movement from left to right, others from right to left, etc., coming from the retina.

I think the main point is that the entire process in a neural network is completely different from how a computer operates. When we name outcomes we may be tricked into thinking it's similar, but when we look at how the output is generated it is a completely different world. That does matter, it has implications for how we think about the whole thing, what we think we can achieve, and how.

If you did this in a computer, imagine not using any storage - not even CPU cache. All data must be processed at once, there are no buffers, not even on an "input pin". You have a stream of data and all you can do is decide where to move it next. It's a horrible analogy but the best I can do right now. Oh, and you don't have a system clock signal, the data is the clock signal. And you don't do any calculations either as a microchip performs them, instead you rely on analog processing: temporal and spacial distribution of the electrical signal matter. For example, if you send a lot of small signals, since they are all actually ions entering the cell (the dendrites of a neuron) it takes time to throw them out again, and if before the ion transporters manage to do that a new signal arrives with more and more of them the amount of ions increases, possibly until reaching threshold (for action potential firing). Same over space: On a dendrite there are many synapses over its length, connected to different neurons (their axons). The charges (ions) can equally build up over space, not just time. So length of wiring matters as well as the shape of the electrical signal - two things we don't want to see having any influence in our microchips. So computation in a chip and in a neural network is vastly different. Computation in the network happens "on the fly" simply by the movement of the signal through the network, encoded as the frequency of an all-or-nothing signal (action potential), but then every analog trick there is is used to decide if and when an action potential fires in connected cells. Actually "storing" values happens over a longer period by changing the connections: New synaptic connections form all the time and existing ones disappear, and existing synapses change ion channel and ion transport channel densities. That is way too slow to have an impact for any given computation, so it plays no roll for trying to catch the ball that's in the air right now.

[hacker42]

You might be interested in reading this: http://www.fil.ion.ucl.ac.uk/~karl/Whatever%20next.pdf

and watching this: https://www.youtube.com/watch?v=HeQfO4byFhg

[joe_the_user]

I have a feeling that you're using an overly narrow meaning for "predict"

Well, I have writing in the context of the article, where people are asked to statically predict the position of an object from watching a video. I would concede it's quite possible that a very accurate object prediction engine lives somewhere in the brain but I think it's even more likely that such information can't be marshaled by a person based on a verbal cues any more than one could verbally describe with accuracy where one's foot should go next while walking.