[tomwalsham]

This is a great overview for people looking to run their own servers, with one of the clearest explanations of DKIM and SPF I've seen. Awesome. As a counterpoint, I'd like to add that from the point of view of an email sender (our company PostageApp is a transactional email service), individual Postfix (Exim, qmail, Exchange...) setups receiving email for small organizations are one of the largest headaches we face.

Large ISPs - Gmail, Hotmail, even Yahoo and AOL to an extent, are predictable. If you play nicely, tick the technical boxes and listen to feedback (SMTP return codes, FBLs, bounces etc) you get great deliverability. Even mid-size ISPs and larger companies usually have some reasonable visibility - responding to postmaster@, internal blacklist checkers, etc.

There are, though, still a nontrivial percentage of organizations and individuals who run their own setups using anything from 1998 'standards' through to modern configurations, combined with other filters like custom SpamAssassin rules, an out-of-date Barracuda appliance, or quirky ASSP installs. They often exhibit some unpredictable behaviours - sending permanent hard-bounce codes for simple inbox-full errors; marking completely innocuous email as spam; requiring three attempts for every email to block spambots (delaying delivery by hours); publishing broken MX records...

Dealing with these can be tricky - even finding the correct admin to contact is often an exercise in futility, all the while, the users are not receiving their critical emails. I guess my message is, when running your own email setup, Caveat Hack0r; if you're not in it for the long-haul, including updates, testing and responding to inquiries, you should really consider going with third-party providers.

[fierarul]

How weird is it that something as basic as email still needs mad hacker skills to set up and dedication to maintain?

Why can't I plug a Linux box to the Internet, have a wizard auto-configure most of it (including DNS stuff, etc), let me toggle some "auto-update" thingy and be on my merry way?

[tomwalsham]

There's an interesting disconnect between the public perception of email transmission and the reality, even from technically savvy observers.

The view you're espousing of the 'basic' nature of email can usually be summed up as : "It's just sending a bunch of ascii from machineA to machineB"

The reality of the complexity of email transmission is that it's an ad-hoc communications network built around an evolution of RFC standards, amended to accommodate i18n, combined with myriad third-party solutions and walled-garden 'standards' to combat a combination of real and perceived threats such as SPAM, DDoS, backscatter, spoofing, joe-jobbing, image encoding...

The main question around your proposed Auto-Updated system is what combination of these solutions are you using, how much are you paying for someone to maintain this, and do you care about the ability to customize for the inevitable false-positives caused by the necessary filters in place.

For large organizations there are solutions - Exchange being one - but they still require large amounts of custom work. The reality of Business A's needs still differ greatly from Business B, even though we're essentially just talking about sending 150Kb of ascii from machineA to machineB.

[fierarul]

Basically, I want to see the Google Chrome of email servers.

[dredmorbius]

Setting up an email server is mad simple.

Setting up an email server that can send and receive mail from other email servers reliably ... not so much.

Most of that is a result of spam and viruses, and the various ad hoc methods which have been institutionalized to deal with both. More often than I'd care to admit, it's the collision between two sites countermeasures and defenses: PGP-signed mail rejected for having attachments, VERP mail rejected for failing to provide a single consistent envelope sender (who but who even looks at envelope sender?), your random residential DSL / server-farm server winding up on a DNSBL, Yahoo pretty much insisting on properly-configured DKIM to receive mail (hey, it's their standard), dictionary spam attacks on your server, remote sites which scan for viruses after accepting your mail (and then blame you for when they don't receive messages).

It's a constant game of whack-a-mole. For basic communications, if you get yourself hosted in valid IP space, it's possible, but if you want high-volume reliable comms, it's getting pretty tough.

[16s]

How would a wizard know what MX to use or how many, their priorities, etc. in DNS? That's just one example. Email requires knowledgeable systems and operations staff. Many small to mid-sized companies do very well selling email services to others. You can purchase such a service, but even then, you or someone working for you has to understand MX records and DNS and why they are important, how to set them up, etc.

[andrewflnr]

I believe the GP's point is that email should be designed such that it doesn't require all that crap.

[debacle]

Because of SPAM, phishing, and malicious emails.

[fierarul]

Web browsers seem to handle malicious web pages just fine. Why is email so special?

[tomwalsham]

Primarily because the Web is a Pull medium, whereas email is Push. To get my eyeballs on the web you first have to trick me into going to your location. To get my eyeballs on Email you just have to know my easy-to-guess personal address.

[guelo]

OTOH, those "critical" automated emails turn out to not really be that critical most of the time.

[dredmorbius]

My alert mails -- pretty damned critical. That's another place where services such as PagerDuty really shine. They get the mail (or pages, or calls) through.