[imtringued]

Inference consists of a single forward pass, which is easy to compute. Meanwhile training has both a forward pass and backward pass (back propagation). The minimum required precision for training is higher since you want to be able to have both fast and slow rates of change. Bfloat16 is preferred for training, while int8 tends to be good enough for inference.