I actually use the 418 code in a simple API we use for IOT devices. I wrote a small program that runs on Raspberry Pis that checks a HTTP endpoint and turns power on and off to a room depending on whether the room is booked out. I wanted a status code on the API that could only possibly be generated on purpose, so if it receives a 418 response it knows to turn off the power; any other response, failure etc turns the power on (so we don't have a room out of use if it breaks).
I solved this with a WeMo plug, which gives me an app and alexa control. Used for electric hair iron of wife, and coffee pot as backup plan with timeout setting, and also saving us driving back home for the "did you unplug the iron" moments.
Both appliances actually have their own auto-off, but I like redundancy.
Honestly the main reason it works like this is that it was my first foray into Rust, and I found JSON parsing a bit of a pain, whereas matching status codes was very simple. If we expand it to return different options in the future then I'll probably rework it.
I think I was having trouble with string lifecycles due to how I was doing something, and at the time I didn't understand them (I'm still not super sharp on it if I'm honest) and just opted to take an easy way out. Lifecycle notation might be the one bit of Rust I find ugly, actually.
I should really revisit this little program at some point, but it's so stupidly simple and works so reliably that I haven't had the inclination to yet.
Why is 418 the particular target? Just because of RFC2324?
> IANA should also typographically distinguish “Unassigned” and “Reserved” in the registry descriptions, to prevent confusion.
This I can get onboard with. Honestly, I've never seen a 418 easter egg, but I'd think it would be an HTTP spec violation if it didn't at least conform to the higher level 4xx definition (Client Error) :)
Nope. Actually, there was a paper published recently on using a phone's gyro / accelerometer (through the HTML5 APIs) in order to keylog users; because the API is precise enough for you to detect subtle motion of the phone as you press on the software keyboard.
Apparently in some mobile browsers, you can continue to poll the API even when your tab is not on focus.
Oh, that's really nice. So if you have a fix (and even if you don't) you can use dead reckoning and tie any points where the GPS is accessible and then re-create the path the phone took. That's a bit of a leak. Wonder how long after or before a GPS fix this would be effective, those phone accelerometers probably aren't all that accurate but you might be able to calibrate the one in a specific phone if you have data available for both of them for some stretch of trajectory.
Even using calibrated and temperature controlled consumer level accel/gyro sensors for dead reckoning results in estimated velocity error reaching few meters/s in a second or two.
Current consumer level sensor quality is enough for:
a) attitude estimation;
b) smooth interpolation between GPS updates if they arrive often enough;
IIRC the accelerometers are way to imprecise to correctly detect starts and stops. Errors compound so quickly that any kind of useful accuracy for dead reckoning is a long ways out.
The use of smartphones is becoming ubiquitous in modern society, these very personal devices store large amounts of personal information and we use these devices to access everything from our bank to our social networks, we communicate using these devices in both open one-to-many communications and in more closed, private one-to-one communications. In this paper we have created a method to infer what is typed on a device purely from how the device moves in the user’s hand. With very small amounts of training data (less than the size of a tweet) we are able to predict the text typed on a device with accuracies of up to 90%. We found no effect on this accuracy from how fast users type, how comfortable they are using smartphone keyboards or how the device was held in the hand. It is trivial to create an application that can access the motion data of a phone whilst a user is engaged in other applications, the accessing of motion data does not require any permission to be granted by the user and hence represents a tangible threat to smartphone users.
Because Mark Nottingham (co-chair of the HTTP WG) first tried to get libraries/stdlibs to remove support for 418 (as part of builtins, since it's not a formally standardised code, he had no issue with application-specific support, everyone is free to return whatever status codes they wish) and faced some pushback, he switched to the alternate tack of getting it into a standard.
It's a response to this story (https://news.ycombinator.com/item?id=14987460), where a contributor has been going to many different projects and attempting to remove language support for the 418 error code.
EDIT: Ah, so it's the same person. In which case it's more of a continuation, and I anticipate that the intention is to make implementations that allow 418 non-compliant.
Note that the "contributor" (Mark Nottingham) is also the author of the document in OP, and deserves a lot of credit for taking this approach. Given that his area of work is HTTP standards, I think he initially started removing 418 to tidy up and make things compliant and isn't just some bore who hates fun. Now he's managed to make everyone happy in a standards-compliant way.
It was interesting reading the threads on /r/webdev about this whole debacle. The threads were full of people saying that he's an arsehole with too much time on his hands who must hate fun.
Nobody actually stopped to read the guys bio and realise that he's more than qualified to be bringing up these issues. It doesn't help that the guy who made save418.com is literally 14 (which seems to be about the average age of /r/webdev).
The /r/webdev thread (https://www.reddit.com/r/webdev/comments/6stdcj/http_error_c...) remained quite respectful actually, afaik there were no real hostile remarks directed towards Mark Nottingham within it. In fact, it's ironic - the very comment I'm replying to is probably more distasteful and antagonistic than anything that could be found in the Reddit thread.
He probably confused it with the /r/programming thread [0], where the top comments with hundreds of votes were stupid insults like "What a dick","miserable bastard", "the most boring man in the universe", "standards committee troll", "he had fun once and hated it", "Soulless, joyless, heathens":
Often reddit threads read more respectful after-the-fact; the less respectful comments either get downvoted enough that they aren't displayed unless you look for them, or are moderated away. Often they are much harsher early in the thread. I'm not sure if that ways the case here, but it is common enough that I wouldn't doubt it.
Why should it matter that the guy who made save418.com is literally 14? Let's look at what he's accomplishing - in spite of his age - rather than whether or not he is substantially older or younger than the opposition in this argument. It seems like that's a pretty cheap argument to bring to the table.
It's weird though. The origin spec is specifically a discrete protocol called HTCPCP, not HTTP, so it has nothing to do with HTTP standards. They can do whatever they want with 418 in HTTP.
They can, but they believe in rough consensus and working code and in being conservative in what they do and liberal in what they accept from others.
It's clear that a lot of real-world HTTP implementations are using HTTP 418 to mean I'm A Teapot, whether or not they should, and that reassigning it would cause practical difficulties.
That's not clear to me. Nobody relies on the feature, it's a tiny piece of code, only some vendors support it, and it's not even part of the spec. (It also can't be reassigned because it was never assigned to begin with)
If they like 418, they can add it to the spec. But this idea that it's been "consumed" is weird. It's like saying <blink> was "consumed" in HTML because two browsers used to support it. It was just an unofficial feature of popular vendor software, and is now officially disavowed.
Basically what we're saying is we're not going to implement 418 in the spec, but at the same time, we're not going to let anyone use it?
...and seems aware that this is simply "tidying up" as opposed to someone from the "No-Fun League"... see the working title: draft-nottingham-thanks-larry-00
Many API developers misuse/overload HTTP status codes when they should actually be using application specific status codes delivered via a wrapper to the response.
Any time an API returns a correct response the HTTP response code should be 2xx. Many developers use 4xx status codes to indicate things like data validation errors or other things that are not part of the HTTP transport.
HTTP is the transport layer, and any application specific scenarios are best handled with a custom error namespace that can be returned within any 2xx HTTP response.
Generally speaking, if the HTTP layer is returning 5xx you have a server problem and client code should not know anything about that or do anything but retry. If the HTTP layer is returning 4xx then the client is likely poorly designed or misconfigured.
But for typical client operation, when there is no server error or client design/configuration error, HTTP responses should be 2xx or 3xx and any additional detail should be handled in an application-specific way, not by overloading the meaning of HTTP response codes, which are for transport-related concerns only.
You should never, ever, ever return a 2xx status code with an error payload. The 2xx series means "Success" and you're abusing HTTP to use it to mean anything else.
With regard to application layer vs transport layer, the 404 Not Found, 406 Not Acceptable, 409 Conflict, and many others errors are specifically application layer codes. If you can't get a more specific code, 400 Bad Request with an expressive payload is useful. The 500 codes are generally transport layer.
The reason that APIs are getting easier and easier to understand and use is because we're using the same design patterns consistently and repeatedly. Therefore, the understanding you gain from the Twilio API can be applied to Stripe. If you use existing tools in new and "innovative" ways that are contrary to - instead of in addition to - to their intention, you're setting back adoption and making your users re-figure out basic things. Don't.
Or worse, people don't realize how you're misusing the tools and build flawed systems.
(The one exception I'd give you is 202 Accepted which is generally used for "we've accepted this for now but processing and final validation may occur later" - it's a tentative success message.)
> You should never, ever, ever return a 2xx status code with an error payload. The 2xx series means "Success" and you're abusing HTTP to use it to mean anything else.
If the transport was successful, then it was an HTTP success.
> 404 Not Found, 406 Not Acceptable, 409 Conflict, and many others errors are specifically application layer codes
These are "application specific" to the resource only, not to the API as a whole. Nor are they particularly informative which is why the spec advocates incorporating a more specific error message. At best, these codes describe a class of errors, possibly useful for logging, but not sufficient for a nontrivial API's error semantics.
My point is not that the HTTP status codes don't mean anything, it's that they are misused to describe application level stuff and used to handle application layer flow control in a way that defeats the purpose and makes them more confusing than they are worth much of the time.
> because we're using the same design patterns consistently and repeatedly.
This is absolutely not the case, every team overloading HTTP status codes seems to do it their own quirky way.
The worst part is when people start bikeshedding about the subtle meaning/intent of HTTP status codes (as your point about 202 illustrates). The simple solution is to define the relevant application layer errors for a specific API and use the HTTP status codes for transport related stuff. It should not matter what 418 or 202 means because the subtle meanings they get int he context of one API/application should not be relevant.
Uh what? If the request was appropriately formatted, not too large, and generally acceptable to the server from a transport perspective, why shouldn't it succeed with a 200?
Error handling is an application level concern, not a transport level concern.
HTTP status codes lack the nuance required to inform the end user about errors in any non-trivial application. I personally enjoy the OP's solution. I've worked on dozens of REST APIs in my life, and I always prefer those that contain a response envelope complete with metadata and error responses.
Pre-emptive edit: When I say error handling is an application level concern, I don't mean "application" from an OSI model perspective, which is narrower in scope than what I'm talking about.
Your slides illustrate a custom response payload which happens to use codes that numerically match HTTP status codes. You also use a response message, I think we actually agree. I'd recommend using a different numerical namespace to avoid any possible confusion about the semantics of those codes. Just because they sound the same to one person reading the spec doesn't mean they will make sense to others.
If the server is expecting data that it doesn't receive (because the form field was left blank) then can you really consider the request "appropriately formatted"? Is it in any way meaningfully different from the case where the server receives a request on /api/invalid-url/ and doesn't have a corresponding resource on its end?
I disagree. HTTP is a widely known standard with a ready to use client/server implementation in basically any platform. Leveraging that semantically in API seems to improve productivity all around. Are there any major trade offs?
The alternative is making standards for how to format the payload and try to get everyone on board while also trying to manage another half a dozen or more legacy implementations of non-standards. Or don't make standards and live with the fact that every API will do its own thing and you have to handle everything differently each time you have to talk to anyone.
As a practical example, I would really prefer not to have to parse a XML response to get a status code in my js frontend, nor would I'd like to examine it via string/regex matching.
No, HTTP is the application layer. TCP is the transport layer.
> and any application specific scenarios are best handled with a custom error namespace that can be returned within any 2xx HTTP response.
No, generally application errors should be 4xx or 5xx codes; greater detail can be provided in the payload, sure, but a proper error condition should be returned, 4xx for errors resulting from the client request being unacceptable in some way, 5xx for other errors (which are, necessarily, server errors.)
(If you are tunneling another application-layer protocol over HTTP, your argument makes sense, but that's not the general case with APIs.)
> (If you are tunneling another application-layer protocol over HTTP, your argument makes sense, but that's not the general case with APIs.)
I'd argue that that is actually what most APIs are doing that are not purely "REST" operating naively on resources.
As we move up the protocol stack, one is the transport for the next. HTTP is not really the application layer protocol, it's a transport for a protocol defined by the API, at least for any API nontrivial enough to benefit from its own specific error/response codes.
agreed... especially because it is an error and if i return a 2xx then the frontend code will then have error logic in the success logic why have two checks when i can have one..
I'd go with 409 Conflict, but I'd also accept 400 Bad Request
Incidentally, the RFC for 409 makes it quite clear that this touches the application layer:
> Conflicts are most likely to occur in response to a PUT request. For
example, if versioning were being used and the representation being
PUT included changes to a resource that conflict with those made by
an earlier (third-party) request, the origin server might use a 409
response to indicate that it can't complete the request. In this
case, the response representation would likely contain information
useful for merging the differences based on the revision history.
In any case, getting pedantic about what's "an error" and using 200 OK for everything that didn't fail at the transport layer is a super frustrating experience regardless of whether or not it's semantically correct. Please don't do that to consumers of your API.
Versioning a resource, per WebDAV is a very specific application.
> using 200 OK for everything that didn't fail at the transport layer is a super frustrating experience regardless of whether or not it's semantically correct
The semantics of my application's protocol do not necessarily mirror the semantics of HTTP, nor are the descriptive statuses semantically similar, since most HTTP statuses are simply about transport (even though a few touch on the "application" of URIs as resources).
Transport layer to me means TCP/IP. HTTP sits above that, in the application domain. If a user sends an invalid request then the response should be in the 400 band. If the server failed in some way to deal with the request (e.g. the database is unavailable), then the response should be in the 500 band.
I would typically decide what my response code should be depending on what I think the response should be.
If a user tries to register a new account then a successful outcome would probably be a new user resource. If the user tries to register a username that already exists, I'd also probably go with 409 Conflict as the user is trying to create a resource that already exists.
Would it kill you to send back a 400 status code upon an error? Having to check for { isError: true } or something similar is obnoxious. I don't see the value gain of responding 200 OK when the operation was not successful.
Some APIs I've seen just use 400 for all generic client-side errors, including request syntax errors, impossible requests, duplicate requests, etc.
I would argue that most of the time, for any sufficiently large application, you'll need to use application specific status codes anyway (as you said), so why bother trying to be specific with the HTTP error codes? Certain client-side applications parse out if the response is a 2xx, 3xx, 4xx, or 5xx, and log it differently. At which point you just need one of them to trigger the different logging behavior.
The only special case I can think of is 401, which you need to send to trigger the basic authentication pop-up window for most browsers.
A big portion of the improper overloading of HTTP status codes comes from WebDAV status codes seeming appealing when in fact WebDAV is a very specific set of functionality that is not really analogous to the way most REST APIs work... notably WebDAV offers locking semantics.
HTTP is not a closed protocol. There will never be a single document defining all HTTP methods, status codes and such. Nothing in the definition of status code 422 makes it inapplicable to non-WebDAV applications. It is as much standard HTTP as status code 400 is (both are “proposed standards” in IETF terms).
> It's possible there is nothing between the OS handling TCP and your application.
This does happen, and it results in the client having to have ad-hoc code to deal with a nearly infinite number of possible "Server error" scenarios, some of which may be normal functions of the API and others which may actually be server failures.
> It's possible there is nothing between the OS handling TCP and your application.
I'm not arguing that one shouldn't use HTTP status codes, I'm arguing that they should be used only for the standard meaning of the code, one should not need to consult a table provided by the API designer to determine which 4xx HTTP status codes warrant a retry, and which 5xx status codes are normal vs exceptional situations.
Suppose you make a call to an API and it returns 5xx because you provided a parameter value that is valid in the URL or payload but invalid in the app (suppose your app doesn't allow usernames that are profane words, for example). The response should not be 5xx, it should be 2xx with a message indicating the application's preferred range of values. 5xx means that something went wrong on the server trying to fulfill the request, 2xx means that the request was fulfilled properly, and any additional info relevant to the client should be passed along as part of that 200 response.
> This is also against the concept of REST
Very few APIs can be designed well as 100% REST. Lots of very typical scenarios necessitate jumping through a lot of hoops to use REST in a pure way. REST fights with database normalization in some cases and with many authentication scenarios. It can also require the client to maintain a fairly elaborate (and brittle) representation of server-side state that is mostly unnecessary.
REST makes sense when there is a perfect alignment between REST verbs and data flows, but when you get into a situation where it's necessary to make multiple API calls to do one logical operation, it would probably have been simpler not to use REST in the first place and to have just designed a simple, clean, non-REST API.
> Suppose you make a call to an API and it returns 5xx because you provided a parameter value that is valid in the URL or payload but invalid in the app
That is wrong. Client mistakes should get 400-level error codes. A 200-level code indicates the request completed successfully, which it didn't.
There is no difference between an "HTTP server" and an app. They are often the same thing, like when using Apache to serve files from disk.
> I'm not arguing that one shouldn't use HTTP status codes, I'm arguing that they should be used only for the standard meaning of the code
Of course you should use the error codes correctly! Don't return a 200 code when the request failed.
> Don't return a 200 code when the request failed.
The request didn't fail, it simply followed application logic and returned a successful response indicating the nature of the application logic to the client. That is not an error condition, it's just application logic.
Every web server I've ever used returns a 500 server error when the transport succeeds but it encounters an exception in application level code. Are you really proposing that's incorrect behavior? I also can't imaging a use for 403 Forbidden and 401 Unauthorized that isn't "application level" logic.
If you think about it pretty much 99% of requests the server receives could be correct if the "application logic" was implemented differently. You can't really decouple the two in any meaningful way.
If you want to return 200 OK status codes all day when why are you using HTTP in the first place?
So then if my application logic catches a thrown exception because I don't want my app to crash every time a server bug is exposed, I should be returning a 200 OK response rather than a 500 Internal Server Error, because the app logic accounts for handling exceptions and returns a response.
I can't disagree more. It just seems like clients would have a nightmare of a time debugging client-side code against such a system.
While I'm inclined to agree with you the makers of SOAP don't seem to share your opinion. Any SOAP fault must be send with an HTTP 500 according to the spec.
My take on SOAP's approach is that it isn't overloading HTTP status messages, it's attempting to impose SOAP's message structure on things that are legitimately HTTP 5xx responses, so that the soap client can handle some of those failure modes and the programmer doesn't have to engineer another protocol for dealing with transport errors or parse a different message format.
Notably, SOAP is not just doing REST it is attempting to offer an abstraction for dealing with transport issues.
Most people designing a REST or rpc style API do not attempt this breadth in their API design, instead they just overload HTTP response codes (from the HTTP and webDAV specs) to handle application-specific stuff, which ends up resulting in the codes being effectively meaningless across applications.
SOAP, on the other hand, is meant to be used in the same way across applications. So if your app uses SOAP to consume multiple SOAP APIs, and you engineer failure handling for the HTTP 500 associated with some kind of SOAP fault, chances are you can reuse this logic with all of the SOAP APIs.
On the contrary, many REST APIs that overload HTTP statuses do it in some arbitrary way defined by the team working on the API, so the consumer can't make generalizations about the meaning of the error codes in the common cases where one might determine (for example) a retry to be necessary.
> Most people designing a REST or rpc style API do not attempt this breadth in their API design, instead they just overload HTTP response codes (from the HTTP and webDAV specs) to handle application-specific stuff, which ends up resulting in the codes being effectively meaningless across applications.
If you look at most of the status codes they pertain to the transport itself: content negotiation, content size, etc.
In other words, the client would be pleased to see one of those codes when making a request and wondering "Why is this not returning the data I expect?".
Compare this to a typical scenario in an application of signing up a user. If the user types in a password that does not meet the password complexity requirements or chooses a username that is already taken, there was not a problem with the transport mechanism, there is an application-specific constraint that the programmer calling the API did expect to encounter some of the time, because of course both of these are common scenarios.
If for some reason the application code does not limit usernames to a small number of characters and the API returns an HTTP 413 status, that is unexpected to the programmer, so the non-200 code is appropriate. The programmer needs to rethink the transport assumptions assuming that super long usernames are OK.
The distinction is that HTTP errors apply to HTTP itself. Are the basic requests and responses functioning properly, etc. But application-specific things generally belong in a separate layer of metadata.
I've seen some very very bad API designs that totally misuse HTTP status codes, largely influenced by the old jQuery pattern of a success and failure callback, with 2xx triggering the success callback and 4xx and 5xx triggering the failure callback. How many times has some server error resulted in confusion because the error callback is being used to handle routine flow control in the application code.
I'm relatively agnostic about this, but having developed integrations for many APIs I generally prefer those that use HTTP status codes + error payload for client side errors, rather than 200 OK + possible error payload, as they're much easier to deal with on the client side.
I'm somewhat interested in spec writing and legal matters. It seem to me, that this rfc is overly verbose given the change it proposes. And (I think, correct me if I'm wrong) rfc:s are generally supposed to be as short as possible.
For example, "IANA should also typographically distinguish “Unassigned” and “Reserved” in the registry descriptions, to prevent confusion." seem like a good idea but is an unrelated matter.
Also it's fun :)