[zahllos]

I can only contribute my own experience. I have a dedicated server with an IP address in a datacentre, with approx 6 users using my email server for their primary email. DKIM/DMARC/SPF all configured correctly. I also have policies that suspend logins for accounts if they send too much in a certain timeframe, because this is a pretty good indicator of account compromise. The limits would never be hit by humans.

I've had three issues. The first was delivering to outlook.com, but this was temporary and resolved relatively quickly: I simply contacted their support. At the time, they didn't bother to validate DKIM or DMARC according to their headers.

The second was a sender sending to us with a misconfigured SPF policy. I had quite strict rules that spf failure => user's junk folder that I had to relax, but I also had a discussion with the admins at the sending company to explain the issue.

The third was yahoo. For reasons known only to them, they decided that IPs they've never seen before will be blocked by returning an smtp deferral that is permanent, which is bad for legitimate mail servers because the email remains stuck in the mail queue forever. I ended up discussing this with their support also and after some discussion that block too was removed.

That's pretty much it. I receive dmarc reports now from many providers so I've an idea what percentage of our email is quarantined or rejected (none). I've been running email since 2011, for my own main email and a few others. I don't think deliverability is that much of an issue and I was able to resolve all the problems I've had in 10+ years of doing this by emailing support, explaining myself and asking to be unblocked. Usually this simply resulted in "OK but if you do bad things we will block you no guarantee of inbox delivery etc etc etc". That's fine. It seems that there is a large degree of per-account spam filtering as well at the big providers mapping to individual users' preferences.

Of course, if you don't set up SPF/DKIM/DMARC, or you have an IP with poor reputation (you can check the DNSBL) or worse a residential address, you will have trouble. I would generally look for a provider that has a relatively strict acceptable use policy, and in particular doesn't allow VPN endpoints to be run from their infra for your email, to reduce the chances your IP has a terrible reputation with the big providers. Also, join all the sender programmes, set reverse dns, don't let your users do things like send bulk email and that will reduce many of the problems.

[LinuxBender]

The third was yahoo. For reasons known only to them

I had to work around them for some datacenter mail relays. The only solution I found was to sum up the number of mail relays behind a SNAT and then apply rate limits for their domain to not exceed 6 concurrent connections total per SNAT. To your point and AFAIK they do not document this anywhere.

[zahllos]

I'm fairly sure they were abusing the standard with this particular technique. Deferred messages are OK if they can be retried later and that's what the MTA will try to do. Permanent deferral I suspect is really supposed to mean "we can't deliver right now and we don't know when but keep retrying".

What this does not do is trigger "undeliverable mail returned to sender" messages, so the end user has no idea their message is stuck in their own MTA's mailqueue until the MTA decides it has tried enough and gives up, and MTAs will usually persist for quite some time.

Spammers won't care what error code you send them or even worry about deferral messages, which is why the temporarily deferred spam trick works (first send is deferred for 1 hour, if you are a genuine MTA and you try again respecting this, it works). But permanent deferral, as I say, is very user hostile. The user thinks they've sent an email, but it isn't in the spam folder of the recipient. The sysadmin then has to go and dig to find out what exactly happened, and remove the mail from the mailqueue.

Luckily so far as I know we are only emailing a single yahoo address.

tl;dr the technique they are using is designed to handle the case where the receiving MTA is offline temporarily. There is a spam defence trick you can use and I don't object too much to that, but they used it to implement their block list rather than outright rejecting, and set the timeout to deferred indefinitely, which is just bad.