[ugvgu0oiua]

One skeptical way of looking at it is that the the explosion of "data science" and ML is basically comp sci running into modeling space in a way it never had before, and getting into territory that it wasn't equipped to handle.

It wasn't that long ago that there were posts on here about statisticians giving conference keynotes about how data science was basically old wine in new bottles, and were being ridiculed for being behind the times, etc.

Now we see that basically no one actually knows what's going on. My guess is when the dust settles a lot of things will be explained, but it won't be as different from established statistical and information theory as some would make it out to be. That is, some of this is new discovery and figuring out new territory, and some of it is neglecting basics that have been there all along.

My guess is the next phase of this is basically comp sci ML research rediscovering mathematical statistics and information theory.

[WastingMyTime89]

> My guess is the next phase of this is basically comp sci ML research rediscovering mathematical statistics and information theory.

That’s the trick students doing ML courses for their CS degree don’t get. The people leading ML research today are statisticians.

Most of them learned their chops before neural networks became cool again. It’s extremely hard to publish anything good without a very solid background in maths.

There is no large divide today between ML and statistics where people could be forgetting things. It’s very much the same field. The main issue is that statistics were already a somewhat immature part of mathematics even before the ML craziness.

[Isinlor]

ML is actually overthrowing some truism from classical statistics.

Classical statistics did not predict double descent phenomena.

The whole idea, that you should not have more parameters than data, was wrong.

[ugvgu0oiua]

I think the story with that isn't finished yet, and in some ways is the perfect example of what I mean.

[keithalewis]

Bingo! Who remembers Symbolics and their lisp machine? Moore's law, not theoretical advancement, is the real story.

[awild]

I took a course on machine learning for my masters. I had trouble understanding most of it, and a lot of trouble solving the coursework (lots of training models), because I didn't get the theory. The kid who had an insane gaming rig always had the best solutions and his understanding was by his own declaration not much better than mine, he could just train way faster so he could make more mistakes and had the better trained models in general.

One of the techniques we learned in the course, was by the account of the Prof, working extremely well, and well documented, its just that nobody really understood why it worked and people were trying to prove why it got optimal solutions.