Yes. And write something to the logs, so you can find out there's a bug before users call your helpdesk. Arguably that's the most important part of any 5xx error.
It's a bit of a purist argument, but 4xx series means the client can fix something and get a better response.
Hmm, I don't think it's that purist. All my HTTP clients have the logic that 4xx is a non-retryable error, i.e. if I repeat the request I will always get the same response unless something is changed, whereas 5xx means that something is wrong on the backend and I should retry it later.
It is true I have often encountered inconsistencies in implementations but I do think that when that happens it really is a backend bug that should be fixed.
There are some 4xx codes that should be retried. 408, 413, and 429 at the very least. There are also cases where it makes sense to retry 404, such as when polling for a resource that is being created and will soon be available.
You are right about the 429, of course, and my more "advanced" clients take it into account and only reissue requests after the requested backoff time. The smaller "stupid" clients error out, as they should, because a dumb retry would just cause "angrier" responses from the server.
I am not sure about the other two though.
413 definitely does not seem retryable, if the client continues sending too large content. Special handling would be needed to break down the data into smaller chunks and when I encounter this error I just break it down to smaller chunks beforehand so built-in handling is not needed.
Never encountered 408. It might be because I never keep connections too long...
It's a bit of a purist argument, but 4xx series means the client can fix something and get a better response.