[Matthias247]

Why TCP over websockets? You can just use the HTTP bodies as a stream in both direction. Which means the proxy just has to strip or add HTTP headers before forwarding. The overhead afterwards is 0 -> you just write to the socket.

[noway421]

Hmm, this choice is indeed strange, websockets still are blocked in some restrictive set ups (squid?).

But still, what is the way of doing stream in both directions? Do you mean opening multi-part form data for uploading and transfer encoding chunked for download? But that would be 2 tcp connection for 1 tcp tunnel. And I believe there's no other way to do it without the overhead of HTTP request/response headers.

[Matthias247]

Technically HTTP represents a bidirectional stream of arbitrary data in both directions, which follows a set of headers and is optionally finished by a set of footers. This is for example quite obvious when you look at the HTTP/2 specification. There is no need for the bodies to be sent in a particular order (response body after request body) or in a particular encoding (form-data, chunked, SSE, etc.). It can be two arbitrary byte streams.

This functionality is exposed by lots of HTTP libraries, e.g. the Go, node.js or C# HTTP libraries will allow to simply read or write to a request/response body stream just like to can write to a socket. As a proxy - just copy the data from one stream to the other.

What are are probably thinking is that the browser environment does currently not allow to use HTTP for arbitrary streaming. Instead they defined some fixed use cases and encodings for these use cases. If you use SSE encoding you can't send bytes without overhead anymore and can only do streaming in one direction. But: You will get a nicer browser API for retrieving the data. Using HTTP bodies for arbitrary streaming will be allowed in future revisions of the fetch API (which e.g. give you a ReadableStream).

Other HTTP libraries also do not allow for streaming but expect the body in either direction to be a fixed length thing. This will allow to represent the body e.g. as a "string" or "byte[]" instead of a Stream from which the application has to read. Thereby the API gets simpler, but not all possible use cases are enabled.

[noway421]

It's true that underlying tcp connection can be reused for virtually anything and it doesn't have to be conformant with having http requests/response. But it would probably get blocked by a proxy which wouldn't be able to pass it down the line.

[rusk]

Not necessarily ... you would have to issue a HTTP request per uplink chunk but HTTP can use connection pooling so that does not necessarily translate to a single TCP connection [0]

Agreed it's not quite as straightforward as the parent poster suggests. I can see issues with this approach for realtime/streaming applications but for applications relying on a similar request/response flow of traffic it would do the job.

[0] https://en.wikipedia.org/wiki/HTTP_persistent_connection

[josephg]

Yes, and for various reasons those chunks can be reordered. And a misbehaving proxy might also try to cache them. And to send data back from the server you need long hanging gets, which can also be subject to timeouts and weird chunking by misbehaving proxies.

These problems are all solvable, but you need to treat http requests like datagrams and (basically) reimplement TCP on top of HTTP.

We've done this several times now. I made one[1] myself a few years ago based on google's browserchannel implementation (that was first written for gchat inside gmail. It supported browsers down to IE5.5). But the best is probably SockJS - https://github.com/sockjs . IIRC Its written by some (ex?) vmware guys, and its great.

But all this stuff is pretty outdated now. Misbehaving corporate proxies are (thankfully) getting much rarer - especially if you tunnel your traffic over HTTPS.

These days you should just use websockets directly.

[1] https://github.com/josephg/node-browserchannel

[noway421]

Isn't connection pooling just using several connections to download one resource using content-range?

[rusk]

Connection aggregation?

[jpillora]

I remember trying run SSH (the current TCP multiplexing protocol) over straight HTTP and found there were issues so I added WS to the mix. There probably is a way to do it though it does the job as is.