[recoiledsnake]

Google's spam filter seems to be having more issues these days. I used to ignore the spam filter, but got a reminder email which forced me to check the spam folder, and lo behold there was the email. I always open emails from that person and Gmail marks it as important, I wonder how it could possibly classify it as spam(assuming the email servers remained the same). The funny part was that just a few days prior I read a comment on HN warning about the spam filter and I had been meaning to check the spam folder but didn't.

[rabidsnail]

I've been getting increasingly fed up with gmail, but not quite to the point of making me set up exim. Does anybody know of a mail server I can throw on ec2 and forget about?

[DanBC]

The answer to "Gmail drops some wanted messages into the spam folder" is definitely not "set up exim" nor "set up [insert email server here]".

There's a reason that people running servers at that level (from that time) are called BOFH.

[pyre]

  > (from that time)

Are you trying to imply that 'mail servers' are some sort of ancient devices whose era has come and gone?

[lmm]

The days when every company maintained its own mail server are gone (and on the whole we're better off).

[pyre]

As we converge on a handful of major email providers, are we really much better off? Would you be singing the same song of Hotmail a decade ago, or is it just because Google seems to be the Monopoly with a Heart of Gold?

[lmm]

If hotmail gave better service than your typical bigcorp's email servers (which is subjective) I'd be saying the same thing. I'm not too worried about a monopoly on email hosting, just because there's very little lock-in; it's trivial to set up as a new email hosting company, and almost as trivial for companies to migrate.

[sbierwagen]

Many blacklists include all the EC2 addresses: for obvious reasons, they get used to send spam a lot.

[overbroad]

Do you think that is a smart idea? To black list entire netblocks? I mean, can we assume that every person using EC2 for email is sending spam?

It would seem to degrade the usefulness of EC2 for anyone wanting to run their own mailserver.

[doesnt_know]

It doesn't really matter if it's "a smart idea", it already happens and you have no control over it. A simple search for setting up your own mail-server is just a search for people who have done that saying "don't ever setup a personal mail server because it's guaranteed to be blacklisted".

Spam has lead email to basically become this closed ecosystem. If you don't use one of the already established major email providers, ISP's or domain name registrars the reliability of email hits the floor.

[overbroad]

While I understand your sentiment, I respectfully disagree that it does not matter if it's a smart idea. Because if it is not a smart idea then that means we can do better. One of the projects I'm working on solves the "closed ecosystem" problem. The use of the term "closed ecosystem" is ironic because it seems to me that the "open" nature of email receiving (not sending) is what leads to the spam problem. In other words, I do not see the problem as the fact that people can send mass quantities of junk email. I see the problem as the fact that daemons accept and deliver mail from anyone. (And then resort to blacklisting.) What if the system was "closed" by default and instead a sender would contact the receiving SMTP daemon directly (no internediary) and would first need either a means of authentication (i.e. he has been pre-approved) or a way to have his sending address revieved and then receive permission to send. Right now you can see someting like this within a domain. For example, one gmail user might be able to send to another gmail user, directly, as they are both able to authenticate. They both have accounts (private accounts, not some RBL, DKIM or other scheme managed by an interloper) and these accounts can be checked. But if one gmail user wants to send to some non-gmail address, the non-gmail recipient has no knowledge of the sender in the form of an account against which he can authenticate. There's no privity between sender and receiver. Instead third party schemes are used. Such as blocklists for sending.

Consider the idea of running a mailserver than only accepts mail from a predetermined set of sending addresses. What would be the chances of receiving junk mail?

[doesnt_know]

"Consider the idea of running a mailserver than only accepts mail from a predetermined set of sending addresses."

How is this functionally any different then blacklists? That's just a whitelist instead. So instead of new mail severs "quite likely" being on a blacklist, they are definitely not going to be on a whitelist.

And no, it doesn't matter if isn't a smart idea when you aren't in a position to change anything. Even if you have a perfect technical solution to the problem, you still have to convince every existing major provider to adopt a solution that isn't even a direct problem for them.

[snowwrestler]

Blacklists are run by anti-spam zealots who really don't care about what is fair or a smart idea.

For example my employer's mail server--which has been sending legitimate person-to-person emails for years (no bulk)--has ended up on blacklists several times because some blacklist operator decided to black-hole an entire netblock at our ISP.

From the blacklist operator's perspective, the broad effect of the block is intended to cause headaches for a ISP as a form of punishment for allowing outbound spam. Our deliverability (and many others) was just cannon fodder for that fight.

[overbroad]

I wish we could get these anti-spam zealots to apply the same effort to stopping junk postal mail. The history of direct mail is interesting and perhaps instructive. It has been kept alive by those who do the delivery (cf. those who do the sending). I have sometimes wondered if the same might be true for email.

If your emplyer knows its recipients (e.g. business partners) and can coordinate with them to run an SMTP service for recieving and sending messages on a different port, would that solve the problem?

[snowwrestler]

This is not a workable solution because the problem is not based on what port SMTP is running on. Organizations and ISPs voluntarily subscribe to email blacklists because they are desperate for any help in reducing spam volumes. They would filter email coming from blacklisted IPs regardless of what port it came in on.

You're basically proposing a whitelist solution, which has many known problems: it does not scale well; it does not handle new or unexpected email partners; it relies on the simultaneous cooperation of all parties; etc. In this particular case it also relies on spammers remaining ignorant of the new port for SMTP--which seems unlikely.

[DanBC]

If his employer knows the recipient the employer can ask the recipient to either stop using the block list, or to poke a whole in it and whitelist their email.

[DanBC]

> It would seem to degrade the usefulness of EC2 for anyone wanting to run their own mailserver.

People chose whether or not to use a block list. Thus your problem isn't really with the person creating the list, but with the mail admin choosing to use that list to filter email. That person feels it works for them.

Very few people should run their own mail server. Email is, now, toxic. Spammers pretty much destroyed email; especially the ability for people to run their own servers for sending.

For a history of a (perhaps overly vigorous block list) look at SPEWS - spam prevention early warning system - which had a few honeypots and which happily blocked large ranges. The Usegroup news.admin.net-abuse.email has very many threads from innocent blocked users and wingnuts screaming "change your ISP!!"

[overbroad]

Spammers destroyed an open email system that relies on a centrally controlled DNS. Probably because they were among the only ones who learned how email works. We never made the effort to teach the population at large, preferring instead to let email be centralized via "email providers". And now, after decades of spam, we still have people who argue it is the best, or even the only, way to do things.

Spammers did not destroy the protocol or well-designed email servers and clients.

"Very few people should run their own email server"

That mindset is why we have a problem, in my opinion. We have actively tried to prevent people from learning.

The history of block lists is a history of the failure of the "email provider" (i.e. "very few people should run email servers") idea. Of course, anti-spam is a career for some people, so "failure" is relative. They've succeeded in trying to exert control over a common internet capability, for profit.

The internet began as peer-to-peer. There was no "DNS". And there were no "email providers". Everyone had a responsibility to learn how to use the network and the basic services it could provide e.g. messaging. Then some people got some bright ideas about how to make money. "Spammers" were not the first ones.

Enjoy that spam in you inbox. It is the product of ignorance.

[kylemaxwell]

But if you're doing it legitimately, and not bulk emailing, EC2 has proper SMTP forwarders you can use. And if you ARE bulk emailing, they have a service (plus you're not using Gmail for that anyway).

[niggler]

I dont think this is a good idea -- if ec2 goes down (as it has in the past) you risk losing email

[notatoad]

SMTP doesn't just drop messages if the destination server is unavailable. It'll either get held back at an intermediary server until your server comes up again, or else it will be bounced back to the sender.

[ams6110]

SMTP does sometimes just drop messages, it's not a protocol that guarantees delivery

[ars]

That's not true. It should either send the email to the recipient or return it to sender. But it should never just drop the message (except for bounces which can be dropped since there is nowhere to send them if delivery fails).

[danielweber]

I think you and the parent are using different meanings of "guarantee."

Yes, everything in the protocol leads to the fact that nothing should drop an email unless it has passed responsibility of it to another server which has accepted the message.

In practice, lots of times things don't work.

[overbroad]

An SMTP daemon will try to send the message a certain number of times, at a certain interval, then, eventually, it will stop ("bounce"). You can configure these settings if you run your own SMTP service.

[jrockway]

There's a reason why a domain can have multiple mail exchanger records.

[rabidsnail]

I risk new messages getting bounced, but all of my email is backed up on my computer. And I'd archive everything to s3 (which has never lost data to my knowledge) in case both my ec2 instance and my laptop disappeared.

[icebraining]

You can set up a different mail server with a lower priority. I run my own server on a VPS, but have Google Apps' SMTP server as backup for those cases, and it's been working fine for months.

[danielweber]

I have experienced, many times, people sending mail to the lower priority SMTP server despite the primary being fully online.

[quicksilver03]

In my experience, the only ones doing that are spammers which assume that a secondary SMTP has no antispam filters configured.

[overbroad]

Assuming your ISP is not blocking port 25 and your internet address is not on some blacklist you can send mail directly from your machine. No need to use intermediary SMTP servers.

Is it possible that someone people might like to use their native SMTP capability for low volume noncommercial email? Does every person who sends email have some overwhelming urge to send spam? Such that we must place pseudo control over sending email, any email whether commercial or noncommercial, in the hands of "email providers"?

Good on you for running your own service.

[icebraining]

My ISP doesn't block port 25, but since my IP address is technically dynamic, it's on Spamhaus' Policy Black List. That said, my ISP offers SMTP servers for proxying outgoing messages, so I used that for a while. I switched to a VPS because my home server died.

[overbroad]

Gotta love that Spamhaus logic. Spammers use cheap dynamic IP's therefore anyone with a cheap dynamic IP that sends an email is a spammer.

Is it cheaper for you to get a static IP from a VPS than from your ISP?