[com2kid]

Does OpenAI on azure still have that insane latency for content filtering? Last time I checked it added a huge # to time to first token, making azure hosting for real time scenarios impractical.

[shakna]

Yes.

Unless you convince MS to let you at the "Provisioned Throughput" model. Which also requires being big enough for sales to listen to you.