> A neural network computes linear transformations
They compute a nonlinear function of an affine transform. Neural networks are nonlinear functions.