[deathanatos]

> Isn't "HTTP pipelining" just normal usage of HTTP/1.1?

> Anyone that doesn't support this is broken. My own code definitely does not wait for responses before sending more requests, that's just basic usage of TCP.

Yep.

There is some "support" a server could do, in the form of processing multiple requests in parallel¹, e.g., if it gets two GET requests back to back, it could queue up the second GET's data in memory, or so. The responses still have to be streamed out in the order they came in, of course. Given how complex I imagine such an implementation would be, I'd expect that to be implemented almost never, though; if you're just doing a simple "read request from socket, process request, write response" loop, then like you say, pipelined requests aren't a problem: they're just buffered on the socket or in the read portion's buffers.

¹this seems fraught with peril. I doubt you'd want to parallelize anything that wasn't GET/HEAD for risk of side-effects happening in unexpected orders.

[X-Istence]

HTTP pipelining is not normal usage of HTTP/1.1. And it means that if request number 1 fails, usually request number 2 and 3 are lost because servers will slam the door shut because of the lack of framing around HTTP it is too dangerous to try and continue parsing the HTTP requests that are incoming without potentially leading to a territory where they are parsing the incoming text stream wrong.

This is what led to the many request smuggling, its because the front-end proxy treats the request different from the backend proxy and parses the same HTTP text stream differently.

Since there is no framing there is no one valid way to say "this is where a request starts, and this is where a request ends and it is safe to continue parsing past the end of this request for the next request".

Servers are also allowed to close the connection at will. So let's say I pipeline Request 1, 2, and 3.

The server can respond to Request 1 with Connection: close, and now request 2 and 3 are lost.

That's the reason HTTP pipelining is not supported by browsers/most clients.

Curl removed it and there's a blog post about it: https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-...

[mgaunard]

There is clear request framing in HTTP/1.1, it's mandated by keep-alive.

[X-Istence]

There is not, which is what leads to vulnerabilities where two HTTP protocol parsers will parse the same request in two different ways (which led to the HTTP desync attacks).

https://portswigger.net/research/http-desync-attacks-request...

There's a reason why web servers will slam the door shut even when the client requests HTTP Keep Alive because they are unable to properly parse a request in a way that makes it safe to parse a follow-up request on the same TCP/IP connection.

[mgaunard]

The link shows how to exploit certain bugs in some bad implementations.

That doesn't change the fact the protocol itself is quite well-defined.

There is no serious HTTP server that wouldn't support keep-alive, this is just FUD.

[X-Istence]

You are conflating keep alive with http pipelining, they are not one and the same. Keep alive may be supported and servers may claim to have fully parses request 1 correctly so they can be fairly confident request 2 can be parsed correctly, but reading the spec one way or another and that is no longer a guarantee that holds.

Keep alive and http pipelining are supported by major servers, some with bugs or issues, but no clients pipeline requests (at least not the major browsers, curl and other popular tooling).

It’s not FUD, pipelining and reuse of an existing connection is broken in the face of trying to parse text protocols that don’t have well defined semantics and where implementations reading the same documentation provide different results because it’s not black and white, it’s fuzzy around the edges.

[mgaunard]

Keepalive mandates that the TCP connection stays open to parse further requests and send further responses and requires the support of the two mechanisms to distinguish the boundaries between different requests or responses (explicit content length and chunked encoding).

Pipelining is just normal usage of TCP, which is a mechanism to establish two queues of bytes between two endpoints on a network.

There is no difference between sending data before or after having received data from the other party. The two directions are logically independent, even if at the transport level data from one direction contains acks of the other direction.

Now, some servers will start processing requests on a given connection in parallel, and will not correctly synchronize the multiple threads trying to write back their respective response to the client. This is just a bug on the server doing multithreading incorrectly, and has nothing to do with any framing problems in the protocol.

I suppose HTTP/2 supports that use case better, since it can multiplex the concurrent responses, but the correct thing to do is to simply treat each request synchronously one after the other, and not parallelize the processing of multiple requests on a given TCP connection.