[mike_hearn]

Well, as I've said several times on this thread, good MQ libraries know how to reopen connections automatically if they break, backoff, retry, connect to several endpoints and load balance between them and so on. All this is an abstraction layer higher than what HTTP provides, so problems HTTP long polling can have in consumer/mobile use cases isn't necessarily relevant. It's like files vs SQLite.

As for the general issue of connections, that's true for consumer use cases. B2B workloads have far fewer problems with that especially when running in the cloud. If your cloud gives you mobile-quality internet then you have a problem, but again, it's a problem a good MQ implementation will fix for you. Consider the "lessons from 500 million tokens" blog post the other day, in which the author mentioned repeatedly that they had to write their own try/catch/retry loops around every OpenAI call because their HTTP API was so flaky.

And again, if you are behind a nasty firewall then you might find your connection dying at any moment because OpenAI got classified as a hate speech site or something. The fix is to file the right tickets to get your environment set up correctly.