[skohan]

What I understood is that they're saying deep learning relies on understanding neural processing in 3 parts: objective functions (activation functions maybe?), learning rules (I guess like back-prop/gradient descent?) and architecture (I assume network structure)?

So it sounds like they want to use this componentization of neural processing to try to understand biological neural networks better.

[etbebl]

The objective function is the entire function the network is training toward, i.e. in a classification task it's the correct mapping of images to labels. The idea here is that real brains also optimize their weights to compute certain useful objective functions.