[scribu]

The way I see it, the utility of theory is raising the baseline. It guides you away from dead ends, so you don't waste time on approaches that could never work.

It's necessarily high-level, so you still need to learn about specific approaches to get practical things done.

[MathYouF]

I think I'd need an author of one of those (SoTA) papers to walk me through how reading part of a 50+ page theory paper helped them achieve any marginal improvement on their path towards productive contributions for me to really understand or believe it. If I could have even a single example explained to me I think I'd get it.

[mellavora]

Maybe not reading it, but did you consider that writing that paper helped them clarify their thinking on the subject?

And based on that clarity, they were able to then go on and make immediately productive contributions?

[MathYouF]

I'm a bit lost on who the 'they' are and what the 'paper' is in this case: The authors of this paper writing this paper, or the authors of SoTA papers writing SoTA papers.

A third option you may have meant which makes sense to me in context is the authors of this paper going on to write SoTA papers. I actually looked through the published works of the first two authors and didn't find any practical work, all similar very long form math notation heavy theoretic papers, which supports what I initially worried about that there's basically two diverging branches of ML papers, one of which I'm skeptical about the practical value of. I have actually not seen any examples of first authors on theory papers going on to publish SoTA results on applications, seeing such a thing I would also consider pretty convincing evidence of the utility of these papers and the symbiotic unification of the two groups.

I wonder if there's been similar concerns raised historically, like at the advent of electricity, between practical engineers focused on creating groundbreaking applications versus people still focused on theory, and in retrospect what contributions continued to be gained by the theoretical work afterwards in those cases.