>> It's just amazing. Imagine going into our code bases and tearing out entire classes or modules; That wouldn't go down well.<<
It's probably more like removing half the CPU cores and your clever code being ok with this.
Thinking of the brain as a computing platform is never a good metaphor. Neural networks do not have a rigid delineation between instruction-storage, data-storage, and CPU. Every neuron wears all three hats and intertwingles those concepts.
In our defense, they didn't grow to be understood but to serve its purpose. So it's akin to a code base with billions of years in the making, without any good documentation.