[baumschubser]

I like long polling, it’s easy to understand from start to finish and from client perspective it just works like a very slow connection. You have to keep track of retries and client-side cancelled connections to have one but only one (and the right one) of requests at hand to answer to.

One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.

[josephg]

Hard disagree. Long polling can have complex message ordering problems. You have completely different mechanisms for message passing from client-to-server and server-to-client. And middle boxes can and will stall long polled connections, stopping incremental message delivery. (Or you can use one http query per message - but that’s insanely inefficient over the wire).

Websockets are simply a better technology. With long polling, the devil is in the details and it’s insanely hard to get those details right in every case.

[wruza]

you can use one http query per message - but that’s insanely inefficient over the wire

Use one http response per message queue snapshot. Send no more than N messages at once. Send empty status if the queue is empty for more than 30-60 seconds. Send cancel status to an awaiting connection if a new connection opens successfully (per channel singleton). If needed, send and accept "last" id/timestamp. These are my usual rules for long-polling.

Prevents: connection overhead, congestion latency, connection stalling, unwanted multiplexing, sync loss, respectively.

You have completely different mechanisms for message passing from client-to-server and server-to-client

Is this a problem? Why should this even be symmetric?

[josephg]

You can certainly you can do all that. You also need to handle retransmission. And often you also need a way for the client to send back confirmations that each side received certain messages. So, as well as sequence numbers like you mentioned, you probably want acknowledgement numbers in messages too. (Maybe - it depends on the application).

Implementing a stable, in-order, exactly once message delivery system on top of long polling starts to look a lot like implementing TCP on top of UDP. Its a solvable problem. I've done it - 14 years ago I wrote the first opensource implementation of (the server side) of google's Browserchannel protocol, from back before websockets existed:

https://github.com/josephg/node-browserchannel

This supports long polling on browsers, all the way back to IE5.5. It works even when XHR isn't available! I wrote it in literate coffeescript, from back when that was a thing.

But getting all of those little details right is really very difficult. Its a lot of code, and there are a lot of very subtle bugs lurking in this kind of code if you aren't careful. So you also need good, complex testing. You can see in that repo - I ended up with over 1000 lines of server code+comments (lib/server.coffee), and 1500 lines of testing code (test/server.coffee).

And once you've got all that working, my implementation really wanted server affinity. Which made load balancing & failover across over multiple application servers a huge headache.

It sounds like your application allows you to simplify some details of this network protocol code. You do you. I just use websockets & server-sent events. Let TCP/IP handle all the details of in-order message delivery. Its really quite good.

[wruza]

This is a common library issue, it doesn’t know and has to be defensive and featureful at the same time.

Otoh, end-user projects usually know things and can make simplifying decisions. These two are incomparable. I respect the effort, but I also think that this level of complexity is a wrong answer to the call in general. You have to price-break requirements because they tend to oversell themselves and rarely feature-intersect as much as this library implies. Iow, when a client asks for guarantees, statuses or something we just tell them to fetch from a suitable number of seconds ago and see themselves. Everyone works like this, you need some extra - track it yourself based on your own metrics and our rate limits.

[_nalply]

One of them 2001 was that Netscape didn't render correctly if the connection is still open. Hah. I am sure this issue has been fixed a long, long time ago, but perhaps there are other issues.

Nowadays I prefer SSE to long polling and websockets.

The idea is: the client doesn't know that the server has new data before it makes a request. With a very simple SSE the client is told that new data is there then it can request new data separately if it wants. This said, SSE has a few quirks, one of them that on HTTP/1 the connection counts to the maximum limit of 6 concurrent connections per browser and domain, so if you have several tabs, you need a SharedWorker to share the connection between the tabs. But probably this quirk also appllies to long polling and websockets. Another quirk, SSE can't transmit binary data and has some limitations in the textual data it represents. But for this use case this doesn't matter.

I would use websockets only if you have a real bidirectional data flow or need to transmit complex data.

[snackbroken]

> if you have several tabs, you need a SharedWorker to share the connection between the tabs.

You don't have to use a SharedWorker, you can also do domain sharding. Since the concurrent connection limit is per domain, you can add a bunch of DNS records like SSE1.example.org -> 2001:db8::f00; SSE2.example.org -> 2001:db8::f00; SSE3.example.org -> 2001:db8::f00; and so on. Then it's just a matter of picking a domain at random on each page load. A couple hundred tabs ought to be enough for anyone ;)

[_nalply]

Good idea if you host a server, but then probably you also could employ HTTP/2 which doesn't have this limitation.

If you offer an executable where the end-user runs it on their own laptop or desktop, you can't expect them to configure multiple domains, but probably there's a way to work-around that: have multiple localhosts like 127.0.0.1, 127.0.0.2, and so on, but there's still one limitation: if the end-user wants to run it on a home server, this won't work as well. In that case tell him to define multiple names for the home server. Ugh.

[zazaulola]

WebSocket solves a very different problem. It may be only partially related to organizing two-way communication, but it has nothing to do with data complexity. Moreover, WS are not good enough at transmitting binary data.

If you are using SSE and SW and you need to transfer some binary data from client to server or from server to client, the easiest solution is to use the Fetch API. `fetch()` handles binary data perfectly well without transformations or additional protocols.

If the data in SW is large enough to require displaying the progress of the data transfer to the server, you will probably be more suited to `XMLHttpRequest`.

[Cheezmeister]

Streaming SIMD Extensions?

Server-sent events.

[crop_rotation]

Streaming SIMD Extensions seems very unlikely to have any relevance in the above statement, server-sent events is the perfect fit.

[moribvndvs]

You could have your job status update push an update into an in-memory or distributed cache and check that in your long poll rather than a DB lookup, but that may require adding a bunch of complexity to wire the completion of the task to updating said cache. If your database is tuned well and you don’t have any other restrictions (e.g. serverless where you pay by the IO), it may be good enough and come out in the wash.