[amelius]

The difference is that you can reason about why it didn't work.

With deep learning models not so. They are too big to reason about in the same sense as your BST algorithm.

Hence, you need a scientific approach to construct them. I.e., with lots of experimenting, hypotheses, etc.

[ethanbond]

That seems like it pushes it further from science, no? The point of a well-crafted hypothesis is that if it doesn’t bear out, you know that it’s because one+ of your assumptions was wrong. Your ability to continue your scientific inquiry is pretty much == your ability to then identify which assumption was wrong.

[hnfong]

You don't need to do an scientific experiment to tell why your BST doesn't work. "Computer Science" is a misnomer because most of its contents and methodologies are from mathematics (which is used in science but is not a science in itself).

CS uses mathematical proofs. You don't need a computer to execute your code to tell why the BST doesn't work. You can introspect your code and figure out why it works or does not work. If it's correct, CS methodology says you can "prove" that it works (without executing it).

Working with large AI models is like working with an artificial brain. It's as scientific as neuroscience in this sense. You make some hypotheses, tweak some hyperparameters, and get a result, which may or may not invalidate your hypotheses. Nobody knows why. Science is not necessarily about knowing the fundamental "whys" (amateurs think humanity has figured all the "whys" out, but that's a lie). It's about establishing some useful model of how things work.

But it's definitely possible to know why your BST does not work, even without a computer, without empirical testing. That's why CS is not a science.

[amelius]

Not sure what you are saying here. Perhaps an analogy helps.

Psychology is a science. You can make falsifiable statements about the human brain. You will need experiments to build and test theories. It's the same with deep learning.

With computer "science" (and math) it's not the same. You can reason completely about your subjects, i.e. you can determine if something will or will not work just by reasoning, no experiments needed.

For more information on the differences between math and science I recommend reading: https://en.wikipedia.org/wiki/Scientific_method#Relationship...

[swexbe]

Seems like the distinction is mainly between which tools are available to you as a scientist (at least if we stick to comp-sci, math is in a league of it's own). When, or if, we can completely model a human brain, a psychologist would no longer need to perform experiments to test their theories.

Given enough computing power, most theories could theoretically by proven or falsified purely through reasoning.

[amelius]

The point is: being able to run a brain inside a computer is not the same as understanding that brain. If you wanted to build a new brain, you'd have to reach for the tool all the time in an iterative way and hope for the best. Only tools that aid in understanding matter. We have only very few tools that help DL researchers better understand what they are doing. Hence DL is more akin towards science than towards math/CS or engineering.