[keithalewis]

Are they talking about the Borel sigma-algebra generated by the open sets of a topological space? What topology is in their mind? Are tempered distributions functions? How does one compose two tempered distributions? (Hint: you can't, and they never actually use tempered distributions.) This is just mathematical masturbation. Everything is finite when implemented on a computer so there is no need for such dainty mathematical niceties unless you are trying lend credence to pedestrian observations about Chebyshev's inequality.

[julbern]

If not further specified, the topology is induced by the metric or norm of the space to be considered. Tempered distributions are used in Subsection 3.1, resulting from the observation that the Fourier transform of a shallow neural network involves a Dirac delta.

Some mathematical concepts are needed in order to present rigorous results. While one can argue about the necessity and relevance of these results for real-world applications, they at least explain various aspects of deep learning in restricted settings, leading to a better general understanding and intuition.