Imgix service broken, wise to avoid and self host?

[thecodeassassin]

The imgix.com service is down yet again (happened a couple of times this month already). All we use them for is resizing images on the fly, which i could do with a small nodejs script, s3 and any CDN. Does anyone have any comments on this?

[mattbillenstein]

Always the danger of using 3rd party services -- their downtime is your downtime. I try to use few of them.

I always thought their posts were kinda neat about racking odd shaped Apple hardware in the datacenter. But it very much seemed to me like a software problem -- I would think it's much easier to build competing image scaling libraries that do not require Mac hardware and software.

And it made me think that they had just hired like datacenter guys with a passion for Apple kit -- a supreme case of where if you have a hammer, everything looks like a nail.

[mattbillenstein]

Wow, looks like ~7 hours so far? https://status.imgix.com/

[mattbillenstein]

https://status.imgix.com/incidents/gzp8gp36tdhq

[thecodeassassin]

Yeah i'm working on a replacement for this service, this is absolutely unacceptable to us.

[codepodu]

We use Imgix to power all our clients' images. We just deployed https://github.com/agschwender/pilbox on production as a replacement. We had to hack around a few things, and clearly not as performant, but at least it works.

[mattbillenstein]

How does caching work after you resize? I built a similar thing 5 years ago and for what we were doing I think it was just a redis cache where the key was a hash of the source uri and the params for the operation and scaler.

Today I might try to hit the cache from nginx using lua -- and then only call the python thing on a miss.

[codepodu]

We put it behind Fastly. We could have used Cloudflare too, but we were racing against time (and angry customers!). We set the cache timeout to 1 hour, and that's pretty much it.

[thecodeassassin]

Nice! Why did you choose to implement it as a proxy instead of using it as a primary source and just fetch from S3 on-the-fly while caching the generated image? If capacity is the problem then making it HA should solve that issue?

[codepodu]

That's actually what we did. From pilbox's docs: > It is not intended to be the primary source of images, but instead acts as a proxy which requests images and resizes them as desired.

We fetched from S3 on-the-fly while caching the generated image using a CDN we already had at hand. This is a band-aid for the night (we're in India, it's 2 AM here) and not for handling daytime traffic. We're obviously hoping imgix will fix their issues in the next several hours. Even if not, at least our customers get a degraded experience than a broken one.

[thecodeassassin]

Did you try putting something like Varnish in front of it? I think you can do the same with nginx, Varnish is just the first thing that popped up in my mind.

[mattbillenstein]

At that time, it wasn't a major part of the website -- and I had redis on-hand, so I just did all this work in a couple small python functions inside of our main webapp and used redis to cache. It worked pretty well. I was only introduced to Varnish some years later and I still haven't used it extensively.