[tazjin]

Amazon has a difficult time with the HTTP standard sometimes. Last time I had to touch an AWS project we discovered a bug[1] in the C++ code backing a Java library (sic).

They had implemented their own HTTP client, but forgot to add the "Host" header to requests which is required by HTTP 1.1.

Interestingly this client sent requests only to their own services, which means that they either released that without testing it or the backend once accepted faulty requests.

[1]: https://github.com/awslabs/amazon-kinesis-producer/issues/61

[hnlmorg]

It's common for HTTP servers to accept requests without a host header. It's not usually needed by the server unless you're hardening it (I don't class it as a security issue but some security audits will flag it up if you don't force the server to reject invalid host headers) or running named virtual hosts (which is more common than it used to be thanks to SNI but you still often see a 1:1 relationship between (virtual) hosts and IPs). So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem.

As an anecdote, about 15/20ish years ago I wrote my own webbrowser. Obviously something highly rudimentary albeit browsers were much easier to implement back then anyway. I was too lazy to read the HTTP spec (it was a hobby project and I was young and impatient) so a lot of what I did was trial and error. I too wasn't sending a host header but it took long while before I ran into any sites that rejected my HTTP requests. The web landscape was very different back then though and IPs were plentiful but it just goes to show how servers have coded around bad clients for years.

[tazjin]

> So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem

This would still be a red flag, as the service in question is their instance metadata service that provides authentication tokens.

Something that important should be integration-tested with the actual service.

[hnlmorg]

> This would still be a red flag,

Perhaps I don't understand the issue you're discussing but how would the client working on 3rd party services be a red flag when that is the desired behavior?

[tazjin]

Sorry if this was unclear: It's a client that they specifically wrote to talk to their own services, and they're releasing it to their customers as an official way to talk to their own services. It could not talk to their own service.

Their own documentation refers to that library (or did at that point in time, not sure about now).

[hnlmorg]

Ahh I did misunderstand you then. Sorry. Yeah that does sound bad.