[throwawaymath]

True, but that was kind of an afterthought of my point. All continuous functions still comprise a vector space.

[soVeryTired]

I don't see how that helps?

> A neural network computes linear transformations

They compute a nonlinear function of an affine transform. Neural networks are nonlinear functions.