[jey]

This is just a practical approximation to the actual mathematical definition of GELU, which is `GELU(x) := x * Φ(x)` where Φ(x) is the CDF of the Gaussian distribution.

[fluoridation]

Isn't that just erf()?

[kgwgk]

They are related but the error function approaches -1 for large negative numbers. Φ(x) approaches 0 and so does x * Φ(x).