[cramsdale]

Hey Folks,

The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a free tier that enables developers to easily use our platform, others see it as an opportunity for exploitation. Even more unfortunate is that it has a negative effect on legitimate users. It’s a fine balance that has been highlighted by several users within this thread.

Spam filtering is not a perfect science, and we’re constantly tweaking things -- with our customers in mind. This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues (which was also highlighted by a few within this thread). If this isn’t the case email me: [e-mail]. For those asking, “hey, why am I being penalized for being a new customer?” See my previous comment about spam filtering not being a perfect science. Then email me.

We’re here and we want to help.

-- Chris (Lead PM for App Engine)

[fpgaminer]

Just a quick thank you for the free tier that App Engine offers. I use it to provide an educational app for my wife, who's a teacher, that she uses as part of her classes. It's nice to be able to provide that to her and her class, without hitting the bank (the school does that enough...). So, thank you.

[brianmcconnell]

Long time App Engine user here. If you're sending email as part of your app, I'd recommend using Mailgun (or similar services). You can set up DKIM, inbound routes, etc, and generally have more control over how your mail is processed. I never got the impression App Engine really wanted to be in the email delivery business, so if that's important to you, there are tools that are focused on that (and free up to a monthly quota). My $0.02.

[metachris]

I've recently compiled a list of free transactional email services with a feature comparison: https://www.metachris.com/2016/03/free-transactional-email-s...

[siva7891]

I completely agree with your recommendation on Mailgun. Using it for past 4 months, never had a problem with their service.

[manyxcxi]

Love Mailgun. You can configure it to just be a very easy mail provider or use the APIs to send mail and turn on all kinds of things like campaign monitoring, tags, click opens, etc.

The best part for me when using it for customers has been the bounce/click through rate tracking. When someone asks me, "how do I know they got it?" It's incredibly nice to point them to the dashboard and show a less than 1% bounce rate (people putting in bad email is almost always the reason) with a log of every single email sent.

Most my clients get this service for free because their volume us low enough. They have quite a generous free tier.

[wastedhours]

+1 as well, after the changes to Mandrill recently, was looking for a new provider and can also recommend Mailgun (at least, for ease and configurability in set up and testing so far, not yet in production).

[pritambarhate]

Yes, Mandrill should have kept the free quota after DKIM and SPF verification. Now, I will be trying out the competitors (Sendgrid and Mailgun) and if I find them at par with Mandrill then for paid projects, I might use them rather than Mandrill.

[mrcncpt]

For our clients, the appeal of Mandrill is the Mailchimp Template Editor. I think this is where their focus is going and so if you don't need that, then definitely check out competitors.

[pritambarhate]

Out of around 18 projects, in which we have used Mandrill, only in 1 we have used the template editor. (I think only about half went for paid accounts when the project went live.) Most of our need was simple and quick transactional messages (Registration, Forgot Password, etc.). The main reason we chose Mandrill was ease of set up over Amazon SES. (Even though we had AWS accounts for all projects.)

Anyways, it's up to Mandrill to choose who they want to serve. I really liked the service though. Best of Luck.

[elbigbad]

Thanks for the suggestion! I was having issues with SendGrid and had never heard of Mailgun. Took literally 2 minutes to get set up on Mailgun, add the credentials to my app, and send my first email! It's a super small app so don't need anything fancy, Mailgun is perfect.

edit: Not to say Mailgun isn't fancy, I have no idea if it is or not, I can say that it works in minutes though.

[markdown]

Hi Chris,

> This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues.

1. My app is not new. It's been running without issue for 2 or 3 years.

2. My app is not on the free tier.

3. On average, my app sends out under 8 emails a day, and it's done the same for over 2 years.

How your algorithm considers my app a spam risk sure beats me.

I'm missing 11 days worth of quote requests from customers. Are these recoverable? (is there a hidden outgoing-spam bin?)

My app started sending mail again today after I changed the src of an image in it from https://example.appspot.com/images/logo.png to http://www.example.com/images/logo.png

How well do you think email clients are going to like emails with images embedded in them without HTTPS?

[klewandowski]

Hi, if you haven't already sent Chris an email, can you send me the details of your account and I'll get answers for you today? [e-mail]

[markdown]

Thank you, I appreciate that.

I sent him an email 14 hours ago and haven't received a response. I'll wait a day and if I still don't hear from him, will contact you.

[tim333]

You might look at Sendgrid. It's free for those sort of volumes

[jcrites]

Possible reason that email service providers and other email sending services might see an increase in new signups and free tier use recently: http://blog.mandrill.com/important-changes-to-mandrill.html Mandrill in essence raised their price, eliminated free tier, and required prepurchased blocks of sending capacity IIRC

The numbers Mandrill released about that business suggest that they had a large number of low volume senders, who may now be looking for a new home.

[je42]

Isn't there an API to find out why a message was SPAM filtered ?

Or are you afraid that "rogue" applications will use it to produce messages that are SPAM but yet not trigger the SPAM filter ?

[jcrites]

The recent post about Candy Japan's attempt to combat credit card fraud gives a good sense of what fighting fraud and abuse is like: https://news.ycombinator.com/item?id=11431881

As an email service provider it's like that and more, since (1) once an abuser uses your service, they've gotten the benefit immediately and keep it even if their account is discovered as fraudulent later, e.g. stolen CC number & chargeback. (2) Abusive users can directly harm good users such as by harming the deliverability of the overall platform. It's not just bad debt, it's bad experience too. (3) Unlike Candy Japan where fraudsters mostly just wanted to check CC numbers and not actually buy product, email abusers really want to send emails (4) It can be hard to tell good and bad senders apart because some companies with an internet presence aren't email savvy and might make mistakes or might get hacked.

Spam filters are always tough because if you give someone transparency into which actions of theirs that you consider abuse, then they will quickly detect and route around your attempt to block them. (See Candy Japan article) It's pretty easy for a human to guess what might be the sign of their fraud and run a few experiments to see what gets flagged e.g. By comparison a machine learning system might be hard to outsmart, but then it's also challenging to explain and troubleshoot false positives. Hence what's effective is often a combination of machine-learned filters and heuristics along with manual overrides by human judgment.

All other things equal, new users are a lot more likely to engage in fraud than existing ones, and so tend to be under more suspicion. Aside from B2B fraud where companies take out lines of credit and then go bankrupt intentionally, it's uncommon for existing established customers to turn fraudulent - they're already vetted. (Consider: who is more likely to be fraudulent. The first time subscriber to Candy Japan, or a subscriber who has been using it for 12 months and is about to buy their 13th month?) It's not a great experience as a new user to be under suspicion, but if it's temporary and easily overridden by a human it can be a decent trade-off - the need to reach out acts a deterrent to spammers but does not deter legitimate users as much (speaking generally).

[JackWebbHeller]

Chris, you say Gmail tightened up their spam filters... Did this take place in the last ~6 weeks or so?

I've noticed the spam filter on Gmail for Google Apps has gotten significantly less accurate recently, resulting in far more false-positives than usual. Any ideas if this is a known issue? I can only presume it's more accurate overall, but it was definitely a noticeable change for our organisation.

[emdd]

Yep, this has been an unfortunate thing for us as well.