[ftxbro]

It's because speech to text, text to speech, and LLMs all have latency. They are working on it. There were rumors that Apple is spending a million dollars a day on its secret AI. Probably some form of it will go into Siri.

Edit: you can try the gpt4-enhanced Bing, it works pretty well with voice

[throwawayllaq]

I expected TTS/STT to be a solved problem now with decades of work on it.

I can understand LLMs having greater latency but all flagship smartphones have inference accelerators these days.

A good response is just an API away, which Google Assistant already does I think (it doesn't give me an instantaneous answer ever).