[Matthias247]

> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure.

I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trailers.

The only thing that trailer support adds is the ability to send an additional late response code. That could have been added also without trailers. Just put another length prefixed block inside the body stream, and add a flag before that differentiates trailers from a message. Essentially protobuf (application messages) in protobuf (definition of the response body stream).

I assume someone thought trailers would be a neat thing that is already part of the spec and can do the job. But the bet didn’t work out since browsers and most other HTTP libraries didn’t found them worthwhile enough to fully support.

[afc]

He offers two facts that I think explain this well enough:

> Additionally, they chose to keep Protobuf as the default wire format, but allow other encodings too.

And:

> Since streaming is a primary feature of gRPC, we often will not know the length of the response ahead of time.

These make sense; you'd enable servers to start streaming back the responses directly as they were generating them, before the length of the response could be known. Not requiring servers to hold the entire response can have drastic latency and memory/performance impact for large responses.

[Thorrez]

This doesn't match what I see in the gRPC spec. It says every message must be length-prefixed.

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2....

Disclaimer: I don't know much about gRPC.

[akshayshah]

I've spent a fair bit of time working with gRPC, and you're correct - gRPC's length-prefixing makes it easy to detect when individual messages are terminated early. You do still need some way to detect streams that terminate unexpectedly on message boundaries - perhaps you could rely on HTTP/2 EOS bits as evidence of an application-level success, but you need some equivalent of trailers to communicate the details of any errors that occur midway through the response stream anyways.

[candiodari]

First, we need to clarify. The problem is that you cannot use GRPC from javascript (yes, there's unofficial, and sketchily supported, hacks, but read on for why they're required)

He explains the problem that caused this, in his opinion, in the article, but not very obviously. The problem is protobuf encoding. It's key-length-value (key identifies the field that follows, length is the length of the value, value is the value). A message is thus "KLVKLVKLVKLVKLV".

The second thing is that repeating a key is always valid (not just when it's declared as an array of values). If a field is repeated but it's not an array, then the newer value overrides the previous one.

(this was done so that if you have 20, or 2000000, massive protobuf files listing, say, urls visitors used, and you need to combine them, you can just concatenate the bytes together and read in the result as a valid protobuf. Also it means you do streaming reads of any protobufs coming in)

So:

1) you don't know when the message ends because you don't have a length field for the entire message (only for individual fields)

2) you don't know the message ends after a given number of fields, for 2 reasons. A, some fields are optional, and may or may not be present. B, even if all fields were sent, a newer version of an already-sent field might be sent to override the previously sent value.

You're right, of course, that the problem can be fixed by only allowing a single top-level message that is decoded in a nonstandard way (and frankly support for this could be added to the official protobuf libraries and it can be made to work by making this requirement optional ... Even concatenating can still be possible that way)

The problem here is rigid, immovable adherence to their own standard (ie. not incorporating a change like demanding KLV on the top level message in an RPC connection, or having a special concat-compatible, one-field-at-a-time decoding, because an architectural decision made 15 years ago said not to do this. Then make this mode required for the HTTP case)

This was an organisational problem. Not that they don't trust each other (obviously a browser developer doesn't trust many people, security really demands they don't. This is not where the problem is). GRPC failed to consider the fallout of them choosing the nuclear option of just dropping out of browsers in order to satisfy an old internal requirement without modifying their design. They chose this outcome, because they couldn't deal with achieving 99.9% of their aims as opposed to 100% ... and missed one of their main aims, let's say they achieved 50% instead.

[akshayshah]

I understand the points you're raising; I'm saying that gRPC enveloping solves them. Once messages are enveloped, you _do_ have a length field for the entire message.

The article is, IMO, somewhat misleading - it discusses the issue without mentioning gRPC envelopes at all, and it seems pretty clear that envelopes were designed (in part) to address this exact issue.

[vidarh]

Another option is simply to reserve a key (or multiple) for providing stream/transport specific metadata which should be stripped out before handoff to the client, such as allowing you to send an "end" marker. Now you're not depending on the transport layer cooperating. It's not a particularly hard problem.

[Thorrez]

That has the downside that you're now limiting what protobuf payloads you can send. You need to have the inner protocol (protobuf) cooperate with the outer protocol. That also makes it difficult to switch to a different type of payload besides protobuf, since even if you could convince the protobuf standard to reserve a specific key for the outer protocol, you might not be able to convince the standard for a different protocol to reserve that for you. The article says "gRPC is definitely not Protobuf specific".

It's like an intrusive linked list vs std::list. Sure you can do intrusive linked lists, but it means you have to clutter up your object with info about how it's stored. It mixes the layers.

[zimpenfish]

> It is obvious for the peer if a complete message (inside a stream or single response) is received

If I'm reading [1] correctly, you can't distinguish between [repeated element X is empty] and [message truncated before repeated element X was received] because "A packed repeated field containing zero elements does not appear in the encoded message." You'd need X to be the last part of the message but that's not a problem because "When a message is serialized, there is no guaranteed order [...] parsers must be able to parse fields in any order".

[1] https://developers.google.com/protocol-buffers/docs/encoding...

[Thorrez]

Yes, the protobuf format makes the end ambiguous, meaning the end needs to be indicated by the protocol containing the protobuf.

But it looks to me like the gRPC spec says that everything must be prefixed by a length at the gRPC layer. So then it doesn't matter that protobuf doesn't internally indicate the end, since the gRPC transport will indicate the end.

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2....

Disclaimer: I don't know much about gRPC.

[Matthias247]

Yes - that’s the length prefixing I was talking about. I should prevent any disambiguities.

But even without that peers should be able to distinguish between truncated and full messages: HTTP2 allows to finish streams either cleanly or reset them. I’m case of a clean close (FIN) no truncation should be expected. I’m case of a HTTP/2 Reset or a tcp connection breakdown the stream must be treated as potentially truncated.