Well, I understand their purpose may be different, but it is nonetheless having my e-mails ending up in "Spam" GMail boxes which triggered me to write this post ;)
I've run mail servers for decades without configuring them and have never had issues. Reputation is probably the most important (note that my domains and even some of my servers were online before these technologies existed) and it's extremely important to get your DNS right, especially Forward-confirmed reverse DNS (FCrDNS). Strictly enforce authentication on submission port 587 and segregate user submissions from application generated submissions so you can tweak each configuration appropriately. Keep in mind that marking messages as spam involves a complex chain of weighting, so if a minor adjustment gets your messages accepted, you could still be straddling a line and would benefit from fixing the basics. And never launch a server on an IP without first checking it against blacklists (demand a new one if it's listed anywhere).
Reputation is everything, but when you need to setup a new server on a new blacklist-checked IP for (non-spammy) mass mailing, without SPF and DKIM your emails will most likely go to the Spam folder, in 2015.
Of course, those things are not guaranteeing delivery, but they play an important role.
Google is particularly insidious: gmail will happily throw away email (not just mark as spam) to "new" recipients, while your own account, which will usually already have a "relationship" with your domain, might receive email just fine.
I just recently had an issue where I tried to send an email to a someone I'd just met. The cc-part that went to my gmail-account got through fine. He didn't even receive spam. After I set up spf, I successfully sent an email to the exact same gmail address.
If gmail had rejected the mail, there'd be no problem -- then I'd know that I'd have to take action. Quietly eating the mail... not cool.
I wonder how long until the only way to send email into gmail/outlook is to set up routing rules that send email to gmail/outlook addresses by logging in to those respective services, and sending directly, bypassing traditional unauthenticated smtp... presumably setting up one "major" delivery would be enough, as gmail can't ignore outlook.com and vice-versa...
Yes but you can't start from scratch without them and do a moderate amount of traffic.
If you have the same clean ips from pre dkim/domainkeys days then don't lose them, or it may be an uphill battle which I would be surprised if you didn't engage dkim to aid in fighting at that point.
SPF and DKIM are neither necessary for mail delivery nor sufficient to assure delivery.
There is, in fact, nothing you can do to guarantee delivery of your mail once you offer it to another mail server. If the recipient doesn't like it, it will be dropped.
You can do lots of things to help. The most important is to not be a spammer. Don't send substantially the same mail to lots of people who haven't asked for it.
> SPF and DKIM are neither necessary for mail delivery nor sufficient to assure delivery.
This statement, while true, is completely useless. Without SPF and DKIM email providers will view your emails with more suspicion, so that a larger percentage of your email ends up in spam even when it really isn't. SPF and DKIM do not guarantee delivery but they reduce the chance of your email being inappropriately recognized as spam.
It's somewhat true.
DKIM checks the authenticity of a email domain. So it definitly helps.
SPF does barely the same. that's why you got into spam if you not set RDNS OR DKIM OR SPF. Since the other server can't be sure if the server is allowed to send mails with the provided domain.
Mailservers are simple, basically you can send with every domain available, however that won't work since other servers will handle that via SPF, RDNS or DKIM.
If it's demonstrably not true, why is it then that multiple reports of mail going into SPAM folder stop coming in, once I setup SPF and DKIM? While it's true that reverse DNS is probably a bigger factor. Not having these 2 setup is going to increase your odds of ending up in the SPAM mailbox.
I have reverse DNS and SPF, but not DKIM and I don't have that many issues getting my email delivered. I did have issues with AOL, but once I registered as the contact for my IP address with them, those issues went away. I've also had one or two issues with GMail over the years, but last I knew, it was okay (I normally don't deal with GMail addresses that much).
Then again, I've had my domain for 17 years, self hosting everything for 16 years (with the occasional IP change, but I think I've had the same IP now for almost ten years so go figure).
Existence of SPF and DKIM will not necessarily keep your message out of the "spam" folders, but absence of them will significantly increase the chances of being delivered there.
I've been running my own (personal) email for a while now. And for a while, both hotmail/outlook.com and gmail have been eating my mail. Google is enough of an asshole to not report anything to the sending smtp, while outlook.com/hotmail at least gives you an error, so you know they got the mail all right, just didn't like your sending ip.
My ip/domain name was in no (public) black lists, however - when I finally set up SPF google stopped black-holing my email.
After I managed to get hold of admins of outlook.com via (I think, there were a few redundant hoops I jumped through):
Outlook.com/hotmail.com provisionally started accepting my email again. All this without my domain sending any spam the past few years.
Personally I think SPF is rather silly, but apparently it's considered an important filter-knob by certain services. I'd much rather gmail/outlook require valid certificates for smtp, and turn of plain-text, than all these add-on protocols that are supposed to avoid "forged sender"-type stuff.
Then I'd have to move over from cacert to a "real" cert, but hopefully that bar will be easier to clear once letsencrypt is up and running.