[nonrandomstring]

Sutton also raises one uncomfortable question. Where are the limits of "our field". If we follow Sutton's (interesting and thought-provoking) advice, where do we stop throwing away human knowledge in favour of general, bare methods? Shall we abandon expert knowledge? Procedural knowledge? Structural knowledge? Algorithms and data structures? Should the rest of computer science surrender in the face of efficiently calculated matrices?

[visarga]

> Shall we abandon expert knowledge? Procedural knowledge?

Apparently GPT-3 is not capable of multi-step reasoning unless trained on code. So it seems having code in the training set generalises an ability to reason in natural language.

[lucidrains]

well, you are assuming these efficient matrices cannot become superhuman teachers that show us the inner workings of the universe in a more optimal way.