[donnowhy]

this is the kind of thing truly making computer science into a science.

with all this "deep" learning technology, understanding based on principles is no longer required, heck, such 'epistemic-attitude' (i.e. knowledge founded on principles) is on the verge of becoming a burden to be shed out.

why reason what happens with the pixels/singals? why try to reason out what Photoshop is doing in reverse direction?

why bother with all this difficult trial-and-error cognitive work if you can throw "data" and "compute" at let the "algorithm" do the "thinking" (figure out the "model").

so instead of doing computer engineering like we used to do math, now we simply measure the outcomes without really having a theory (and the worrying part) nor bothering to make one; like physics! or chemistry!! just consider modern 'pharmaceutic' research i.e. letting the models duke it out

[Nevermark]

I recently read an essay about this tragedy of algorithms, perhaps phrased differently.

The bittersweet success of general algorithms that just chew up data and compute time to arrive at very good answers, over algorithms crafted from a careful understanding of the problem.

I tried searching for it, but haven't found it yet.

[seanhunter]

This is something known as the “strong search vs weak search” debate. You can imagine generally that many problems can be transformed into a search of a multidimensional problem space for a solution.

Algorithms that are made to a search a specific domain are able to use knowledge of that domain to be more efficient at that particular problem. This is “strong search”. The downsides of this approach include the fact that they won’t be generally applicable outside the domain and also they rely on us understanding the domain well enough ahead of time to build a solution tailored to the problem.

“Weak search” algorithms use no specific knowledge of the domain. So these are things like stochastic methods, ML generally etc. These are weaker in the sense that they will always require more time and compute power than the best strong search in a given domain but are generally applicable and are able to reveal structure in the search space that they aren’t told about (perhaps because we don’t know this structure) ahead of time. So weak search can be used to find “good enough” answers to problems where we don’t yet have an exact/closed form type solution for instance.

This is the same as if you know the formula for calculating the area of a circle you will be able to calculate the area of a given circle much more rapidly than if you use (say) a Monte Carlo simulation to tell you what it is. But the circle method only does circles wheres the exact same MC will be able to tell you the area of any shape.

As compute generally becomes cheaper, weak search methods become viable for more and more problems given often people’s time requirements are not that critical in many applications.

[trabant00]

> understanding based on principles is no longer required

It very much still is if you care about getting the right answer. If you only care about getting convincing sounding bullshit you just ask the "AI".

[Nevermark]

Language models certainly generate their fair share of bullshit. But then, they are actually trained on bullshit - raw human output, much of it informal - so as a first step, it is a sign of great success.

By the time language models don't ever generate bullshit they will have succeeded in understanding and communicating at a level most or all of us will never achieve.

But most models, including photoshopping detection, are not language models. Their job is to perform some well defined problem (classification, pattern recognition, relationship generalization, etc.) better than alternative methods. They perform, or they don't. There isn't room for bullshit.