[overbroad]

Do you think that is a smart idea? To black list entire netblocks? I mean, can we assume that every person using EC2 for email is sending spam?

It would seem to degrade the usefulness of EC2 for anyone wanting to run their own mailserver.

[doesnt_know]

It doesn't really matter if it's "a smart idea", it already happens and you have no control over it. A simple search for setting up your own mail-server is just a search for people who have done that saying "don't ever setup a personal mail server because it's guaranteed to be blacklisted".

Spam has lead email to basically become this closed ecosystem. If you don't use one of the already established major email providers, ISP's or domain name registrars the reliability of email hits the floor.

[overbroad]

While I understand your sentiment, I respectfully disagree that it does not matter if it's a smart idea. Because if it is not a smart idea then that means we can do better. One of the projects I'm working on solves the "closed ecosystem" problem. The use of the term "closed ecosystem" is ironic because it seems to me that the "open" nature of email receiving (not sending) is what leads to the spam problem. In other words, I do not see the problem as the fact that people can send mass quantities of junk email. I see the problem as the fact that daemons accept and deliver mail from anyone. (And then resort to blacklisting.) What if the system was "closed" by default and instead a sender would contact the receiving SMTP daemon directly (no internediary) and would first need either a means of authentication (i.e. he has been pre-approved) or a way to have his sending address revieved and then receive permission to send. Right now you can see someting like this within a domain. For example, one gmail user might be able to send to another gmail user, directly, as they are both able to authenticate. They both have accounts (private accounts, not some RBL, DKIM or other scheme managed by an interloper) and these accounts can be checked. But if one gmail user wants to send to some non-gmail address, the non-gmail recipient has no knowledge of the sender in the form of an account against which he can authenticate. There's no privity between sender and receiver. Instead third party schemes are used. Such as blocklists for sending.

Consider the idea of running a mailserver than only accepts mail from a predetermined set of sending addresses. What would be the chances of receiving junk mail?

[doesnt_know]

"Consider the idea of running a mailserver than only accepts mail from a predetermined set of sending addresses."

How is this functionally any different then blacklists? That's just a whitelist instead. So instead of new mail severs "quite likely" being on a blacklist, they are definitely not going to be on a whitelist.

And no, it doesn't matter if isn't a smart idea when you aren't in a position to change anything. Even if you have a perfect technical solution to the problem, you still have to convince every existing major provider to adopt a solution that isn't even a direct problem for them.

[overbroad]

If Alice and Bob agree to run their own SMTP daemons, closed to the public and not necessarily on port 25, and they each agree to put the other on their "whitelist", how is this functionally different from the current third party controlled system? Answer: 1. Immediate delivery, assuming Bob and Alice keep their machines online. 2. No spam. 3. No third parties exerting control over their mail. No idiosyncratic delivery policies.

I'm afraid there's no need to convince any provider of anything. At this point, Alice and Bob are sending and receiving email without the need for any third party "email provider".

Functionally blacklists and whitelists are the same. They both have the same goal. But they are not the same in their effect. Blacklisting an entire netblock to stop one bad IP address affects many IP addresses who do not need to be blocked. Whitelisting a single known IP address does not have that side effect. For Alice and Bob, handling their own messages may be a desired option. Of course, not everyone may follow Alice and Bob's example. But who cares? The population using email is enormous and diverse. The point is that if someone wants a better solution than what "email providers" offer, she can get it.

[doesnt_know]

Your proposed solution isn't really email though. What you are describing has already been solved by instant messaging/jabber/twitter/facebook PM etc. Some of the solutions that already exist need a third part provider, others don't.

The problem to solve is how do you have a fixed address where anyone can contact you, spam doesn't get though and you don't have to maintain personal black/white lists. This is what email currently provides. Granted, the spam part varies depending on the provider.

[overbroad]

Yes, you defined the problem in the opening sentence of your second paragraph. But I disagree that you should not have to maintain a whitelist. What are your email contacts i.e. what is your email address book? You already maintain a list of people you correspond with, whether you think of it that way or not. And when you want to correspond with someone new, you have to give them your email address. As it stands, there is no _reliable_ way for them to look it up. There is no worldwide directory of email addresses. In fact, what do we do? We try to hide lists of email addresses.

If everyone had a fixed address with a mail server running, "lookup" i.e. simple MX lookup, might be possible, e.g. if your IP address is 1.2.3.4, anyone could send mail to inquiries@[1.2.3.4] or something like that. But I'm not sure that alone really solves the problem.

Email still works without a worldwide directory. People exchange email addresses and they keep lists of them known as address books.

[DanBC]

White listing a single IP address is the functional equivalent of block listing the rest of the Internet?

You claim it's not a closed ecosystem, but it appears to be totally closed and locked off. The only way to get access to it is to be invited in.

> The point is that if someone wants a better solution than what "email providers" offer, she can get it.

Not if they want email.

[overbroad]

Where did I claim it's not a closed ecosystem?

It's starts closed and it is opened by invitation. Yep. That is exactly how it works.

If you cannot understand that approach, then that just means it's not how you think. It does not mean that the approach makes no sense or has zero utility.

Maybe a stupid analogy can be made if we pretend "Facebook" is the internet (of course it's not, but it does present a messaging system so play along for a moment). On the one hand you could make every Facebook user your "friend" and thus able them to send you messages, and then when people abused that privilege - and we know from experience some would - you could block them. On the other hand, you could only make a select number of people who you know and trust your "friends" and thus only give a select number of people the privilege to send you messages. Chances are, they won't try to sell you Viagra.

On the one hand there are times you may want to enable the entire network to be able to send you messages. On the other, there are times you may only want to allow a small subset to send you messages. Not sure about you, but I don't receive important email from all that many different people. People's social circles are only so big. There is a certain carrying capacity beyond which it becomes unmanageable.

[snowwrestler]

Blacklists are run by anti-spam zealots who really don't care about what is fair or a smart idea.

For example my employer's mail server--which has been sending legitimate person-to-person emails for years (no bulk)--has ended up on blacklists several times because some blacklist operator decided to black-hole an entire netblock at our ISP.

From the blacklist operator's perspective, the broad effect of the block is intended to cause headaches for a ISP as a form of punishment for allowing outbound spam. Our deliverability (and many others) was just cannon fodder for that fight.

[overbroad]

I wish we could get these anti-spam zealots to apply the same effort to stopping junk postal mail. The history of direct mail is interesting and perhaps instructive. It has been kept alive by those who do the delivery (cf. those who do the sending). I have sometimes wondered if the same might be true for email.

If your emplyer knows its recipients (e.g. business partners) and can coordinate with them to run an SMTP service for recieving and sending messages on a different port, would that solve the problem?

[snowwrestler]

This is not a workable solution because the problem is not based on what port SMTP is running on. Organizations and ISPs voluntarily subscribe to email blacklists because they are desperate for any help in reducing spam volumes. They would filter email coming from blacklisted IPs regardless of what port it came in on.

You're basically proposing a whitelist solution, which has many known problems: it does not scale well; it does not handle new or unexpected email partners; it relies on the simultaneous cooperation of all parties; etc. In this particular case it also relies on spammers remaining ignorant of the new port for SMTP--which seems unlikely.

[overbroad]

The problem is the open internet. There are ways to establish connections to a peer-to-peer overlay that take this out of the equation. The ISP is unlikely to block UDP traffic on some high port. Throttle perhaps, but not block. That gets us around the port issue. Once you log on to the overlay, the ISP only sees one port.

Then we are free to do our SMTP of other messaging as we desire. Each connected machine can choose what ports it wants to listen on, if any.

And what if this does not need to scale? What if it's only being used for a small group of people? What if all the people know each other? A very specific but very common use case. Not everyone is a celebrity with a gazillion "friends". Nor is everyone constantly conversing with new acquaintances. Some people have old friends and family. So I've heard.

Is it worth the spammer's time to try to find an SMTP daemon for each indivdual email address? Under the current system, things are centralized enough that a spammer can spam hundreds, thousands or even hundreds of thousands of recipients via sending to a single SMTP daemon.

Spammers have to send enormous amounts of spam to be successful. Having to do extra work to find an SMTP daemon just to send email to one recipient, and have to do this repeatedly, seems like it would not be worth a spammer's time. At least, not when it's so easy to just spam people that are using email the usual way: allowing some third party to handle their mail.

[DanBC]

If his employer knows the recipient the employer can ask the recipient to either stop using the block list, or to poke a whole in it and whitelist their email.

[overbroad]

Are you suggesting his employer doesn't know the recipients? Hmm, if true, I wonder what sort of email that might be.

Not the type I'm interested in.

[DanBC]

Read the thread again.

> For example my employer's mail server--which has been sending legitimate person-to-person emails for years (no bulk)--has ended up on blacklists several times because some blacklist operator decided to black-hole an entire netblock at our ISP.

You replied:

> If your emplyer knows its recipients (e.g. business partners) and can coordinate with them to run an SMTP service for recieving and sending messages on a different port, would that solve the problem?

That solution introduces a bunch of problems: you're running more software that's open to the Internet and thus introducing insecurity; you're asking people (who might not be technical) to install and run software and use a different mechanism when they want to communicate with a subset of users.

The other solution is to just ask the people that you're sending email to, but who are using a whitelist / block list to add you to the white list or exclude you from the block list.

[overbroad]

You're making assumptions. About how things would work and about users and what they can and cannot do. Typical online discussion. Lots and lots of assumptions.

I do not understand your last sentence. Didn't he say his ISP is blocking outgoing mail? The recipients are powerless to unilaterally change that situation.

Think about this for a moment. Forget the corporate example. Imagine one user has a daemon listening for mail (no setup, it's all been set up for him:- it's "built-in" to his OS). Imagine there is an authentication method e.g. a shared secret and perhaps even some obfuscation like port knocking to hide the open port. Even assuming a determined spammer can get past this, is it worth his time? He will reach a grand total of one user.

We can even use a small overlay, where the IP addresses are private, not routable on the internet. The spammer needs to get into the overlay network first, again defeating things like shared secrets or perhaps private keys to identify machines before he can even get a shot a access to a listening mail daemon. That's not easy to do if the users stay logged in. And again, if the network is small, with a few hundred users or less, maybe only a handful, is it worth his time?

[DanBC]

> It would seem to degrade the usefulness of EC2 for anyone wanting to run their own mailserver.

People chose whether or not to use a block list. Thus your problem isn't really with the person creating the list, but with the mail admin choosing to use that list to filter email. That person feels it works for them.

Very few people should run their own mail server. Email is, now, toxic. Spammers pretty much destroyed email; especially the ability for people to run their own servers for sending.

For a history of a (perhaps overly vigorous block list) look at SPEWS - spam prevention early warning system - which had a few honeypots and which happily blocked large ranges. The Usegroup news.admin.net-abuse.email has very many threads from innocent blocked users and wingnuts screaming "change your ISP!!"

[overbroad]

Spammers destroyed an open email system that relies on a centrally controlled DNS. Probably because they were among the only ones who learned how email works. We never made the effort to teach the population at large, preferring instead to let email be centralized via "email providers". And now, after decades of spam, we still have people who argue it is the best, or even the only, way to do things.

Spammers did not destroy the protocol or well-designed email servers and clients.

"Very few people should run their own email server"

That mindset is why we have a problem, in my opinion. We have actively tried to prevent people from learning.

The history of block lists is a history of the failure of the "email provider" (i.e. "very few people should run email servers") idea. Of course, anti-spam is a career for some people, so "failure" is relative. They've succeeded in trying to exert control over a common internet capability, for profit.

The internet began as peer-to-peer. There was no "DNS". And there were no "email providers". Everyone had a responsibility to learn how to use the network and the basic services it could provide e.g. messaging. Then some people got some bright ideas about how to make money. "Spammers" were not the first ones.

Enjoy that spam in you inbox. It is the product of ignorance.