|
|
|
|
|
|
by codedokode
18 days ago
|
|
|
Sorry I did not understand the point about underflows. Underflow happens with and without denormals - denormals just allow to go a little further before the number turns into a zero. Also, there are cases when you do not want to crash on overflow - for example, live audio processing, you do not want sound to stop only because there ws an overflow in one audio sample. |
|
|
One option is to throw an exception. In that case a program will never generate any denormals. Denormals can appear in such a program only when they are present in input data generated by another program, which has been executed with a different exception configuration.
The second option is to generate denormals. In such a configuration every underflow generates a denormal and such denormals are propagated through later arithmetic operations, unless they are added to or subtracted from a much bigger number, when they disappear.
What you say about audio is the very reason why denormals have been introduced in the standard. You are allowed to choose the second option, i.e. to mask the underflow exception, so there will be no underflow exceptions and the underflows will generate denormals, which will introduce minimal errors that are acceptable in most applications.
Unlike with the errors introduced by denormals, which are typically negligible, the non-standard third option, to flush denormals to zero, can easily produce huge errors that are unacceptable for most applications. Thus this non-standard option is acceptable only in applications where the errors cannot have serious consequences, e.g. they may cause bad pixels that are not noticed in an animation or they may change the probability of some token in LLM inference, which may not change the actual sampled output, and even if the output is changed, it might not matter among more important causes of hallucinations.