[spxneo]

so how would you go about improving that?

[freediver]

Not using an LLM for it.

[itake]

we only send 0.5-5% of traffic to gpt4, thanks to smaller faster cheaper models. So not all of our traffic is hit with 50s latencies :-/