[callahad]

Postmark just dropped our outbound email for high bounce rates. Falling back to SendGrid :/ Try again in 5 minutes.

Edit: Ask HN: Any suggestions for solving this? Hitting the frontpage means people are spamming the login form with bogus addresses like [e-mail] that bounce and cause trouble for us and the ESPs. Not ideal, but I'm not sure how to solve this for a small scale side-project. Discussion in https://github.com/portier/portier-broker/issues/96

[danieltillett]

I think you have just discovered the fundamental flaw of your approach. My only suggestion is send the first email from your own server and if this does not bounce then send again from postmark.

I used to use a telnet based approach to check if the account exists, but most servers these day don't respond with the correct error response if an account does not exist.

[callahad]

I'm not sure it's a fundamental flaw: it's exactly identical to sending a confirmation email when a user signs up for a website, which is considered a best practice on the Web. If that works well enough for the Web at large, it should work here, too.

As far as I can tell, the trick is building up that initial reputation and doing as much mitigation as possible up front: checking for MX records, rate limiting, soft-failing with CAPTCHAs for things that look suspicious, etc.

I know what I'll be hacking on for the rest of the week. :)

[danieltillett]

I should have phrased this differently - it is the fundamental flaw of all email based authentication systems. Your approach is currently a bit more prone to problems since you are one nasty script away from being bounced out of existence.

Yes you need to protect the log in page by rate limiting, Captcha, looking up mx records, etc.

One approach I have thought would be good to use is a rainbow table like approach. Most people are not very imaginative about the fake email accounts they use.

[jcrites]

One workaround might be to request that the user send an initial email to your servers from their address before attempting the authentication. You'd want to validate that the email arrived and is valid according to SPF/DKIM/DMARC. You might consider allowing only messages that are positively authenticated by one of those technologies (not e.g. vacuously valid for lack of SPF records). Admittedly this will require a more complex user interaction, but it will avoid the abuse problem you're describing. You could make sending this email easy by supplying an appropriately filled out mailto: link; it should be enough to click it and send. You'd then follow this up with your current authentication step.

I've been thinking about this space for a while (e.g. https://news.ycombinator.com/item?id=12411204 ) and would be happy to chat or brainstorm.

[medmunds]

There are a handful of (commercial) email address verification services, which could automate some of the logic for you:

* https://kickbox.io * https://www.emailhippo.com * http://www.briteverify.com

The pricing might be prohibitive for your usage. I don't know if any of them offer discounts/free usage for open source projects. (Not endorsing any of them; they've been sitting on my own list of things to investigate.)

Also, as an aside: SendGrid may not be suitable for transactional email you need delivered consistently quickly, unless you purchase their dedicated IP option. (And if you're bouncing a lot, a dedicated IP could actually be counterproductive for your delivery.) Recent incident [https://sendgrid.com/docs/Classroom/Deliver/shared_ip_thrott...] and discussion [https://news.ycombinator.com/item?id=12142728].

[medmunds]

Oh, also, another resource if you want to roll your own email verification: https://www.scottbrady91.com/Email-Verification/Python-Email...

[jlgaddis]

Do you have a plan for users whose mail server does greylisting?

Having to wait 20 minutes to receive an e-mail so that i can login would be a real PITA.

Also, "recipient verification" (which some others have managed) isn't a good long-term solution.

[kosinus]

That's a really good question, because there's no plan yet. In fact, Portier currently doesn't allow more than 15 minutes for verification.

Lots of services already depend on email verification during signup or password recovery. I'd love to know, have you seen any interesting solutions around greylisting in the wild?

[jlgaddis]

> have you seen any interesting solutions around greylisting in the wild?

To "bypass" or somehow defeat greylisting, you mean?

I'm not aware of any via workarounds, no. I would certainly be interested in hearing about any , though, since I use greylisting on mail servers with thousands of users.

In some cases, such as when using OpenBSD's spamd [0], you (an SMTP client) don't even get to talk to the "real" SMTP server until you've successfully "passed the tests". This is the primary implementation of greylisting that I use, FWIW.

[0]: http://man.openbsd.org/OpenBSD-current/man8/spamd.8

[intimatica]

What about only explicitly allowing domains that have DKIM enabled?

Having that form of validation should help since DKIM was made to specifically stop email spoofing; and anyone serious about implementing Portier would understand that need.

Outlook/Gmail etc aren't likely to be flagged by ESP's for the most part as those have reputation and rudimentary spoofing protection, at least more than [e-mail] anyway.

Persons using custom domains are usually the admins of those domains, and Office 365/GApps/Registrars etc. provide simple ways to enable DKIM.

[jlgaddis]

In response to your other (dead) comment, "SPF" records actually are TXT records.

In the past, there was an SPF RR type but that's been deprecated.

[Canada]

Depending on what kind of bounces they penalize you for, making sure the domain has a valid MX record before you send the mail might help.

Won't help if you get flooded with a bunch of invalid usernames at a big email provider, but as your volume of legitimate volume grows the bad ones should hurt less.

[jlgaddis]

Hopefully the OP already knows this but (just in case) an MX is not required. An A RR will be tried if an MX doesn't exist.

[Canada]

Exactly, which is why validating the MX is better than merely testing for NXDOMAIN. If you try delivering to something with implicit MX it is far more likely to fail than something with proper MX records. It's also more likely to fail because the A record doesn't point to a host running SMTP, so your provider will be forced to queue it and keep retrying. SMTP servers pointed to by stale MX records are much more likely to return permanent failure right away.

Nobody who actually cares about reliably receiving mail should depend on implicit MX. Of all the things mail administrators have to do these days, like reverse lookup, SPF, and DKIM, is it reasonable to care about anyone who can't be bothered with MX records?

I'd gladly reject such domains if doing so improved overall mail delivery rate to popular domains. And unlike 99% of situations where an email fails to show up, the sending web site can instantly report the reason for failure.

[jlgaddis]

You bring up a very good point that I hadn't really thought of before ("... is it reasonable to care about anyone who can't be bothered with MX records?") but that has caused me to stop and think.

I manage e-mail systems with thousands of users and a fair number of domains as well. In most cases, I also manage the authoritative DNS servers for these domains so I make sure that all the appropriate records (MX, TXTs for SPF, DKIM, etc.) are set up properly.

You're right, though. If example.com hasn't bothered setting up an MX record for example.com -- even if mail is hosted on the same machine as identified in the A RR -- they probably aren't worth worrying about. It seems reasonable to conclude that they really aren't too concerned about being able to receive mail.

[eric_h]

briteverify is very good at detecting bogus emails, however it's not terribly cheap.

[djokkataja]

Why not just take the demo page down, at least for now? The concept is very simple.

[scottydelta]

Why dont you add a captcha?