[plus]

I've personally always thought of the Dirac delta function as being the limit of a Gaussian with variance approaching 0. From this perspective, the Heaviside step function is a limit of the error function. I feel the error function and logistic function approaches should be equivalent, though I haven't worked through to math to show it rigorously.

[yuppiemephisto]

All these would be infinitely close in the nonstandard characterization. I just picked logistic because it was easy and step is discontinuous so it shows off the approach’s power. If I started with delta instead I would have done Gaussian and integrated that and ended up with erf.

[thrance]

It is, in a way. The whole point of distributions is to extend the space of functions to one where more operations are permitted.

The limit of the Gaussian function as variance goes to 0 is not a function, but it is a distribution, the Dirac distribution.

Some distributions appear in intermediate steps while solving differential equations, and then disappear in the final solution. This is analogous to complex numbers sometimes appearing while computing the roots of a cubic function, but not being present in the roots themselves.