[seanhunter]

I would argue that there are a few fundamental ways to make progress in mathematics:

1. Proving that a thing or set of things is part of some grouping

2. Proving that a grouping has some property or set of properties (including connections to or relationships with other groupings)

These are extremely powerful tools and they buy you a lot because they allow you to connect new things in with mathematical work that has been done in the past. So for example if the GP surmises that something is a Lie group that buys them a bunch of results stretching back to the 18th century which can be applied to understand these neural nets even though they are a modern concept.