|
|
|
|
|
|
by owlbite
924 days ago
|
|
|
I think in most deployments, they're not fp32 by the time you're doing inference no them, they've been quantized, possibly down to 4 bits or even fewer. On the training side I wouldn't be surprised if they were bf16 rather than fp32. |
|
|