[mach1ne]

Yes, it’s really rather like alchemy in some sense. Stuff works, and often nobody knows exactly why.

[uoaei]

"I don't follow the latest ML scaling and theory research" does not in any way equate to "these things are unknowable".

[lhnz]

Hm, watching Neel Nanda videos recently and I do get the feeling that there are lots of unknowns in ML and also in what trained networks have learnt.