[nok22kon]

that's incorrect in the presence of batching. it's tough work making it truly deterministic:

https://x.com/FireworksAI_HQ/status/2069873437217276015

[vidarh]

It's not that hard. What is hard is making it truly deterministic and retain high throughput.