[amelius]

Wasn't this already obvious from simple networks which implement e.g. the XOR function?

[NhanH]

Simple network without non linear function (eg sigmoid) can’t learn XOR

[nerdponx]

This is completely different. It seems to be about exploiting floating point underflow to have linear nodes perform nonlinear computation.