Hacker News new | ask | show | jobs
by contravariant 2193 days ago
Universal approximation doesn't seem that strong a claim at all, you could use simple polynomials to achieve just that (to some extent RELUs are polynomials as well, provided you use 0/1 or integer weights, but that's probably besides the point).

It would be much weirder for a function with lots of parameters to not have universal approximation in some sense, as it would imply that you lose some degrees of freedom somewhere.