[quicknir]

Statisticians are either Frequentists, or Bayesianists. Fundamental to both of these approaches is the involvement of probability. A technique that does not have a probabilistic interpretation is not a statistical technique. This is also largely related to the difference in goals between statistics and machine learning: statistics is primarily about inference, and machine learning is primarily about prediction. You can predict without necessarily making any meaningful probabilistic statements about data. There's not really much to infer without saying something probabilistic.

[srean]

Neural nets have absolutely been associated with probalistic interpretations. Good resources would be David McCay and Radford Neal. Both their approaches are Bayesian. A far more trivial way to associate a probabistic interpretation is to claim that the neural net is the conditional expectation.

And who says frequentist and Bayesian are the only two views. Where would you shelve prequential statistics then ? Or nonparametric regression

Prediction has definitely been a part of statistics but often, as you rightly claim, as a byproduct. And yes i would characterize the focus of stats and ml exactly as you did

[quicknir]

The point is not whether somebody has ever tried to associate neural nets with probability, sorry if my previous comment made it seem that way. The point is that neural nets are not fundamentally tied to it. You can try to tie them to probability, but you certainly don't have to, it mostly isn't, it isn't mostly taught that way, and the big open problems in the field don't involve it.

Statistics works the other way. You basically always start with some kind of probabilistic model. And then, if you even bother with prediction, you work towards prediction from the probabilistic model. With stats you don't need to interpret or add probability after the fact, it's already there.

Obviously stats and ML are enormous fields, with quite some overlap. And people tend to go after low hanging fruit; if many people who studied neural nets have formal probability backgrounds it simply makes sense that someone will write a paper on it. And I'm generalizing here (same goes with frequentist & Bayesian comment). But there absolutely is justification for saying "neural nets are not really a statistical technique".

[srean]

I agree with a lot what you are saying, but rest assured its colored by personal rationalizations one has made while learning these things and not objective facts.

nns are in no way less fundamentally associated with probability than say linear regression. In both cases you can start with a probability model and derive the final form as a logical consequence, or you can start with the final form and slap a consistent probabilistic model on top.

The main thing is that any test you come up with that carves nns away from stats is going to carve a whole lot of other things that people have no trouble calling them stats. This controversy essentially stems from the need to claim a technique for ones own tribe and not concede to another. I am making no moral claim here just an observation and neither is it a novel one.

BTW I am firmly from the ml camp and not stats. I enjoy poking a little bit of fun at statisticians and try being gracious about their criticism of ml. That said i feel no need to make a groundless claim to a technique when they have no less rights to it in terms of objective claims. Rights steeped in culture, fiat and history are a different matter.

[davrosthedalek]

But isn't prediction by itself a probabilistic concept? Isn't one interested in the confidence of the prediction?