[gradys]

Who's to say that neural nets aren't implicitly learning these concepts as they train? In fact, I'd be very surprised if nothing about the networks' internal state corresponded to linguists' models of how language work.

Nothing about focusing on the math and not the language means that these concepts don't have a role in how the models learn language. Just as nobody is told about parts of speech as they're acquiring their native tongue, we don't necessarily need to explicitly tell machine learning models about parts of speech in order for them to learn how to use that aspect of language correctly.