[SendGridBecca]

SendGrid Support here! We wanted to step in and provide some color for what is going on. We’re currently making changes to our shared IP system in order for our customers to have a better delivery experience in the long term. As a result of this, the IP group you’re sending from has a new shared IP address(es).

As these IP groups warm up, you may see some deferrals if you are a Free or Essentials customer. However don't worry, this warm up period won't last long. Maybe a few days at most, or until major email receivers have enough data to determine the legitimacy of email being sent from these new IPs.

Keep in mind SendGrid will continue to attempt to deliver these throttled emails on your behalf for up to 72 hours (it rarely takes the full 72 hours to deliver an email throttled in this way).

If you wish to avoid disruptions like this in the future, considering upgrading your account to a Pro or higher plan (https://app.sendgrid.com/settings/billing), which includes a dedicated IP address as opposed to sending from a shared IP group. Dedicated IP addresses are great because instead of many different users sending from the same IP or group of IPs, you are in complete control of your sending reputation.

Customer feedback is extremely important us here at SendGrid, and we have made these changes as a result of that feedback. We know in the long run, this will immensely help your sending.

This will go away, just hang in there with us! If you have additional questions, please feel free to contact us by going to support.sendgrid.com.

[samwillis]

We may not be a big customer to you and we may only send a few thousand transactional emails per month but even on the "essentials" paid package we expect a service level that is usable.

Suggesting that it's ok to wait a few days for our transactional email to deliver is frankly offensive. By definition transactional email is expected to be delivered in minutes if not seconds - days? That's got to be a joke?

This response is enough in its own to tip us into make a switch to an alternative supplier. It may be of no interest to SendGrid if an "essentials" customer leaves but if this is how you treat them you may find that more than a few do.

Frankly this entire situation has been mismanaged. As email "experts" you should have been able to migrate you customers over to the new IPs one by one as the IPs send level increases.

Finally suggesting that as this only effects some of your customers and so doesn't warrant being on the status page is just you making excuses to lie to your customers and hide the fact you have a serious issue. If this was a planed move with any possibility of delay you should have notified the effected customers before hand. We only discovered the problem when our customers stopped receiving emails and our support team saw an increase in support requests.

P.S. Your competitor Postmark has a very different oppinion about dedicated IPs https://postmarkapp.com/why/delivery

[linkregister]

Disclosure: I am a former employee of a Sendgrid competitor.

The proper way to transition IPs is to split traffic between them, so when one IP is throttled, traffic can continue. These accounts should have been assigned the new IP in addition to the old one. Once warming completed after a couple of days, the old removed. I would strongly recommend all affected customers to open support tickets with Sendgrid to get put on good IPs.

[medmunds]

> Suggesting that it's ok to wait a few days for our transactional email to deliver is frankly offensive

Another SendGrid customer here, in exactly the same situation (low-volume, purely transactional, paid "Essential" plan). But even if I were sending marketing blasts, a few days delivery delay would be unacceptable. It's hard not to hear SendGrid's response as "we don't really support anything below the Pro plan".

SendGrid has put up a support article about this: https://sendgrid.com/docs/Classroom/Deliver/shared_ip_thrott...

FWIW, a helpful SendGrid chat agent did move us back to a warmed-up IP pool once I complained, and our mails are going through again.

[Edit: clarify we're on a paid plan -- just not the Pro one.]

[samwillis]

Nice to hear they are trying to help some customers.

I opened a support ticket 12 hours ago and have just been told it will eventually be sorted.

[medmunds]

Try the chat option -- it's more immediate. (Though the chat agent did mention they're pretty backed up.)

[0xmohit]

Maybe SendGrid suggests using a courier instead of email for fast and guaranteed delivery of messages.

[chelbell]

My company has been scratching our heads for the last 12 hours trying to figure out what was going on while waiting for SendGrid support to email us back. Without hearing anything I finally found this thread through a Twitter search, thank goodness.

I'm astounded that an email company couldn't do us the decency to email their customers to tell us what's going on. We've wasted valuable time and lost business because of them.

We're switching over to Postmark now. This year has been a nightmare in transactional email land from Mandrill to SendGrid to now Postmark, ugh. Hoping Postmark can do both you and I well.

[andrewvc]

I'm not a customer, but seriously, you could at least apologize for disrupting customer's service.

The tone of this message is pretty terrible.

[0xmohit]

I'm also not a customer.

However, this message warns me not to sign up unless for a Pro account (don't know what that means).

You are essentially telling your Free or Essentials customers to go and f*ck off. Not nice.

[zo1]

Any deliberate maliciousness on their part aside, at the very least they could have "Warmed-up" the new IP addresses before switching customers over. That's like moving your existing customers over to a new server and only then having them wait while you "install software". At the worst, it's malicious. And at the best, just plain incompetent.

[rahimnathwani]

They need to send legit emails from those IP addresses to warm them up. It's not like a car that you can warm up by idling and not moving.

[zo1]

Again, should they not be able to do that before using the customers as guinea pigs? You'd think that they'd have a large buffer of "untainted" IP's that they slowly cycle towards. I.e. they "slowly" warm them up with legitimate emails of their own, and then have them on hand for when they need to replace existing IPs for customers.

[rahimnathwani]

I agree with you.

The point I was trying to make is this: in the example you gave ("waiting to install software") or the one I gave (warming up a car on the driveway), you can get the system ready for action without creating any impact in a production environment.

The case of email servers is an unusual example where you have to do real world tests in order to make the system work. In most other situations, you can do everything before the rubber hits the road.

So, whilst this could have been handled much better, it's not a simple situation.

[mathgeek]

I'm not a SendGrid user, but I'm curious as to if this was announced or sent out as a notification in some manner before it happened.

[airyt]

tl;dr: don't use sendgrid. no it wasn't. i have 100s of messages stuck in my queue marked as 'deferred'. these are critical emails that affect us financially. and the response is to "wait a few days". laughable. even without this process, lots of messages get stuck for 10 minutes or more with no response from their support department as to why. And this is a PAID service. over to AWS i go. i don't need complex reporting or elegant UI. just send the bloody email.

[chrisferry]

If your email reliability is affecting you financially maybe you should pay for a pro account? I would NEVER use a "free" service for e-mail if I depended on it for $ This is a lesson well learned for you.

[airyt]

i am paying sendgrid. i'm not using the "free" service either. Pro is 8x the cost i'm paying now. mandrill was the same price and actually delivered email. go figure. and i'm equally pissed at the horrible level of customer service as i am at the quality of the actual service.

[p49k]

If you're sending financially critical email, you probably shouldn't be using the $10/month tier of an online service. Email is complicated, I feel like most of these types of services explain the tradeoffs of each price point fairly well.

[airyt]

sorry, but i've used all sorts of services from free to paid. this has been the worst experience of all of them. mandrill was best. i'm pissed at the crappy customer service as much as i am about them deferring email (i understand the complexities) and we cant afford $1000 per year on email.

[codingdave]

It sounds like you can't afford to NOT spend $1000 per year on email.

[airyt]

lol

[reboog711]

No notification was sent to my client who is using them.

The response to my support ticket on the subject was "things are working".

[angersock]

Wow, you done goofed here. Where to start?

> However don't worry, this warm up period won't last long.

How long is "not long"? Your customers can't fix support problems this causes by saying "Well golly gee our vendor says it'll be over in not long"--in fact, thanks to you, they may not be able to email their own users at all.

> Keep in mind SendGrid will continue to attempt to deliver these throttled emails on your behalf for up to 72 hours

After which time...what? You'll summarily delete the emails? Refund the customers for failing to do your jobs? What exactly?

> Customer feedback is extremely important us here at SendGrid, and we have made these changes as a result of that feedback.

Sure, but you didn't warn the effected customers. Some of whom were paying you.

~

You all fucked up here, and this is probably going to lose you a lot of accounts. Good work.

[omfg]

Phone support said emails sold probably start going out Monday. But maybe Wednesday. If we switched to a Pro account it'd still be a few days for that IP to 'warm up'. We had no choice but to switch to Mandrill.