[ma2rten]

Lack of theory is actually mentioned as one of the issues in the presentations.

I don't think your examples are good though, Max polling reduces noise. RuLU learn faster than Sigmoid or tanh.

[kmicklas]

> I don't think your examples are good though, Max polling reduces noise. RuLU learn faster than Sigmoid or tanh.

That's not theory, that's just observation of the results. Why should we expect it to work that way?