[cmehdy]

This sounds like the opposite of what Richard Sutton seemed to advocate for in his "Bitter Lesson"[0]. I don't know nearly enough to advocate for one thing or the other, but it is fascinating to see that those approaches seem to compete as we venture into the unknown.

[0] http://incompleteideas.net/IncIdeas/BitterLesson.html

[fizixer]

They're not the opposite, and both are correct.

Sutton is saying 'over a slightly longer time'.

You can wait 20 more years and super-duper-deep-NN-on-steroids, and hardware a million times as big and powerful, would rediscover all of theoretical physics.

Or you could inject some theoretical physics acquired by humans and make DNNs smarter today.

[cmehdy]

I assume your 20 years is a guesstimate, and I do think it misses the point of what Sutton's writing is. The trap here is that there's always to be more computing in the future, so where do we draw the line? The idea is to think differently now, for the pursuit of actual progress down the road. Which, by the way, is exactly what people were doing about 40 years ago and what put down more than the foundations for all the tricks we're pulling these days.

[fizixer]

I see what Sutton said as a "statistical learning and artificial intelligence" researcher in line with what the authors of the physics paper presented as "an application of learning research to computational science and engineering, CSE, surrounding physics".

CSE researchers did not sit down and wait for AI researchers to learn the bitter lesson before they resumed their work.

CSE research goes on independent of whether AI/GOFAI/ML has a winter, a summer, an ice age, or a global warming.

It just so happens that in light of the recent progress of AI/ML, specifically 2012 to 2019, they see the utility of incorporating a tiny bit of ML to their vast array of methods.

The paper shared in this thread is merely another attempt to advance such an incorporation. If it doesn't pan out, they go back to doing CSE on physics without any AI or ML.

[cmehdy]

That makes sense. As you said, those two sources don't have to be contradicting each other if they complement instead.