[mmcclimon]

> Normally, this means that spf.messagingengine.com has its own SPF DNS record, which will probably list some valid IP addresses that emails can be sent from. There's also one for Mandrill, for transactional emails. You can have as many of these as you want.

This isn't strictly true (that you can have as many as you want), because SPF has a (IMO) very silly hard limit of 10 DNS lookups per record. From RFC 7208:

> Some mechanisms and modifiers (collectively, "terms") cause DNS queries at the time of evaluation, and some do not. The following terms cause DNS queries: the "include", "a", "mx", "ptr", and "exists" mechanisms, and the "redirect" modifier. SPF implementations MUST limit the total number of those terms to 10 during SPF evaluation, to avoid unreasonable load on the DNS. If this limit is exceeded, the implementation MUST return "permerror".

We see this not infrequently at Fastmail, when customers report DMARC validation problems, and the answer turns out to be that they've got too many includes in their SPF records, so SPF always fails.

[technion]

That limit of ten is extremely easy to meet when someone casually says "Hey we've started using Freshdesk for ticket tracking, setup DNS please". Ok so you include:email.freshdesk.com. That record itself includes four other freshemail.io DNS lookups, and sendgrid.net, which includes another one. So you're seven DNS lookups in just for that. Gmail's recommended include:_spf.google.com includes four more, at which point the limit is exceeded and SPF is broken.

[mmcclimon]

Yeah, for sure! Outlook is also very bad about this, and usually when we see it people have added include:spf.messagignengine.com (Fastmail's SPF, which is all IP addresses) and one other include, which does a bunch of recursive lookups. This is often very difficult to explain to people, and there's not even really a way to work around it, short of using different domains for different sending needs (which many people are unwilling or unequipped to do).

[Hello71]

why can't you just periodically fetch and cache the records yourself? i.e. an ALIAS record for SPF. if the customer doesn't want to use dynamic dns, then you could even provide this service for them, of course charging a premium for the "enterprise feature".

[marcbradshaw]

There are services which do that, but it doesn't work in every situation. It can't (for example) cope with SPF macros, which is where the server looking up the record can replace parts of the record it is requesting with variables from the SMTP session (for example, by inserting the connecting IP address into the record lookup domain). In short, as stated in the article "email is complicated"

[sillystuff]

> why can't you just periodically fetch and cache the records yourself?

The only way this will help you is if you recursively de-reference all the records, and replace them with a list of allowed sender IPs. Now you have a new problem. You only get 255 characters in a single string in DNS. You can chain multiple 255 character strings together into a single record though, so you can get up to 4K characters. But, if you have a bunch of authorized spoofers (e.g., mailchimp) that different departments of your organization uses, even 4K characters may not be enough (source: I went through this exercise a couple years ago and our organization would have needed closer to 8K characters for the de-referenced records).

Our solution is to include our own outbound MTAs, and ?all. On inbound, we don't use SPF as a signal at all, as there are too many sites with misconfigured records.

Long term, we are trying to get each department to send spoofed mail using per-department subdomains, e.g., [e-mail], [e-mail], etc.

DKIM + DMARC is designed to handle the above without any of the issues SPF has.

[dillondoyle]

I didn't know this! Is there an easy tool to check how many chain-on dns lookups happen?

I also typically have at least 2 for clients, including evidently google's 4. so this is important to know!

[teddyh]

> Is there an easy tool to check how many chain-on dns lookups happen?

I use https://www.kitterman.com/spf/validate.html

[kcolemangt]

https://fraudmarc.com/spf-record-check/

If your spf is too heavy, add universal spf and then watch it work by either sending a test message or rechecking your spf with https://vamsoft.com/support/tools/spf-policy-tester

[dillondoyle]

thank you!!

[kcolemangt]

> they've got too many includes in their SPF records, so SPF always fails.

Any domain can toss "v=spf1 include:UniversalSPF.org -include:x.UniversalSPF.org" in front of their broken SPF record to automagically clean and fix it. This authorizes mail that the domain owner expects to pass, and fails the mail they expect to fail.

Source: One of my startup's covid projects was creating and giving away https://UniversalSPF.org. We'd already been providing SPF Compression commercially since c. 2015.

It's free to use and already trusted by several hundred businesses.

Here's a good spf evaluator in case you want to see universal spf fix your domain's policy: https://vamsoft.com/support/tools/spf-policy-tester and a more technical deep dive for you other command line geeks: https://fraudmarc.com/introducing-universal-spf/

[aj3]

Outsourcing security is a very stupid thing to do. Of course copying a line you don’t understand into your DNS isn't that much better but at least you control it.

In this case configuration is handled by an unknown entity that you no contractual obligations to you. Don’t do it.

[metafunctor]

How does it work?

Based on a cursory inspection, it looks like UniversalSPF makes use of a custom DNS server and the the %{i} and %{o} macros for SPF.

Before this, I wasn't even aware that SPF had macros!

[hoseja]

Quick, somebody prove it's Turing-complete!

[kcolemangt]

It works by inactivating spf terms that cause errors then serving out a cleaned up and compressed policy for you on the fly.

[zinekeller]

Holy, I thought that there was a draft to change the limit to 10 different base domains, it seems to have died.

[kcolemangt]

Not to overly shill my own free solution here but quite a few people use Universal SPF (a little string you put in front of a traditional spf record) to overcomes the 10 dns lookup limit.