Hacker News new | ask | show | jobs
by commandersaki 266 days ago
Email for me is a critical service, and the reasons I stopped self hosting after about 15 years is:

1. Because I couldn't ensure consistent backup and restore with regular monitoring,

2. no disaster recovery plan and in doing so it'd be more expensive than going through another email provider,

3. not always on top of security (my friend that I colo'd with also ran an email server and his system was struck with ransomware (with no backup [except a copy of email via thick client] or DR); I seemed to get away unscathed because I was using FreeBSD which generally less of a target).

I agree that it is little maintenance, but once you're off the happy path, it can be a huge pain in the arse and devastating.

2 comments

DR: MX and retry

email has easily one of the best responses to failure modes ever and its ancient!

Most smtp daemons will put outbound emails in a queue and run the queue. If the other end is unavailable then it will generally retry on a schedule with some sort of increasing period and then give up after a week or so.

You can easily define multiple inbound relays via your MX records which predate SRV and generic TXT and are supported everywhere.

I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.

Google and MS and Co really don't screw you around if you follow the rules and that largely involves only SPF being compulsory and the rest (DKIM n that) are nice to have. If you do send spam then you will be crucified and rightly so.

Email is not a critical (its important) service because of course you have several other means of communication starting off with the SIP n RTP server you also run ... 8)

I agree with that aspect of DR; I guess I was more thinking of availability, in that I can probably handle a few hours of not receiving emails, but if it goes longer than a day or so then I'd be pretty miffed. Like I said it's all doable, but it requires a lot of effort, and is probably best not left to someone running a one man show, and once you have more than one person you likely now have to deal with trust and expenses.

I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.

Again, so have I, and as I said the happy path is always easy, it's when things go wrong, and I'm not even talking about IP reputation or any of the usual issues that people bring up running email.

Email is not a critical (its important) service

Really depends; I still have many services such as banking where I need to use auth codes, also a lot of security is tied to my email in terms of private comms and recovering services.

Suppose your email service went down and the people you run email for complain, do you tell them "oh don't worry it's not a critical service, you can still communicate over other mediums"? Would that work for say gmail?

I had a client domain banned by Gmail due to a missing DKIM, even though they had fewer than 1000 emails per month and SPF was correctly set up a decade ago. The bounce message explicitly said they are bouncing because DKIM is missing.
I suspect there is more to this than meets the eye.

I have had a Gmail email account since they were invite only back in the day and I run my company email system and by my company I mean my (ie MD) so I'm quite keen on it working.

I recently migrated the whole shebang to MS365 from Exchange on prem. I have kept our MX records pointing to our on prem SMTP daemon (Exim). That means that I can redirect mail to mailboxes as I wish - I am not beholden to MS. Several addresses end up being delivered to an on prem imapd (Dovecot).

Anyway, I did set up DKIM when it was invented and then DMARC and then I ditched them because it messed up with mail lists. That has all been sorted but I still don't have DKIM on my company domain.

I have never setup DKIM on my personal vanity domain collection. The only recent fix I had to carry out was to fix up reverse DNS (PTR record) for an SMTP/MX address. That is proper old school and only one recipient domain even noticed and dropped mail.

The bounce message you received may have said DKIM but it may have been lying or simply that was the last thing that went wrong or whatever.

The big email systems are run by reasonable people who do not discriminate against well run tiddly email systems. They will absolutely crap on spammers inbound (despite hosting them) and IP reputation is king. There are a lot more rules too and it is rare that any transgression is final - pretty much all systems are score based rather than absolute on one failure.

I had my email banned by Yahoo because I would get rate limited and I didn't have a way of surfacing those messages (huzzah opensearch)

What got me out entirely was when I attempted to send an email to a colleague at a random ass no name university and my email was flat out rejected with no way to reach out to the administrators. I wouldn't have cared if it wasn't such a unique project (oil and gas exploration using ML). I have not self hosted email (in earnest) since that day over 10 years ago.

> Email for me is a critical service

This is exactly why I only trust myself to do it. I almost lost my gmail account a couple of times in the past, and every time it was quite stressful. Since then, I use gmail as a backup email provider, than is, pretty much never.

Due to the way mail servers work, you have a couple of days to sort out your troubles before you will start missing emails. At worst, you can always buy Google for Work or some other SaaS and point your MX servers there.

Backup is always a hard problem, but I got to live with Hetzer Clould backing up my VMs, Hetzher Backup boxes as restic backup targets and a tiny Celeron server in the laundry closet for local backups.

This is exactly why I only trust myself to do it.

In theory that makes sense, one thing I specifically omit as to why I stopped running my own service is in the past in a bout of paranoia due to the onset of a mental condition, I literally rm -rf'd my laptop, including a lot of files that were unrecoverable. Thankfully I didn't do this to my server at the time. Even though I've been stable for a long time, all it takes is a relapse (or even just a lapse of judgement) and boom your servers (and backups) become vulnerable.

I also don't trust that I can secure my systems and backups better than a company that dedicates itself to running a service for multiple users and have dedicated security/infrastructure teams. Sure I've never actually had an issue, but as with the anecdote of my friend, it just takes one failure. Also economies of scale helps with security; it is easy for an attacker to exfil or do damage to a smaller corpus of data (few to no customers [users]), than a large corpus of data across 1000s of customers.

I wouldn't trust a free service or a service that doesn't provide adequate support such as Microsoft or Google, but there's obviously a good selection of email providers out there that do an excellent job, much better than those self-hosting because they work with economies of scale.