|
|
|
|
|
|
by nmfisher
16 days ago
|
|
|
I'm not a regular v2.5 user, so I don't really know. But given the TileRT team write-up says the entire network gets quantized to FP8 (and experts get quantized to FP4)[0], I'm assuming there's at least a modest drop in quality. [0] https://www.tilert.ai/blog/breaking-1000-tps.html |
|
|