[mmoskal]

Their tech report says one inference deployment is around 400 GPUs...

[fspeech]

You need that to optimize load balancing. Unfortunately that gain is not available to small or individual deployment.