[lddemi]

Author here. We do and did use autoscaling heavily but at a certain scale we just ran out of headroom on the smaller instance types we were using. Jumping to a much larger instance types meant that we will likely never run into those headroom issues again, plus solves other problems like faster spin up, better sidecar connection pooling and allows for a much higher hit rate on per instance caching.

[TekMol]

Did you consider switching form CPython to Pypy?

[latchkey]

You were autoscaling a single threaded process. You had 1000 connections coming in and scaling 1000 workers for those connections. Everything was filtered through gunicorn and nginx, which just adds additional latencies and complexity, for no real benefit.

What I'm talking about is just pointing at something like AppEngine, Cloud Functions, etc... (or whatever solution AWS has that is similar) and being done with it. I'm talking about not running your own infrastructure, at all. Let AWS and Google be your devops so that you can focus on building features.

[aeyes]

According to the article they have a monolithic Django application so this will have at least a couple of seconds start-up time. That is not a good match for Cloud Functions.

Django also has in-memory caches, for example for templates which can be extremely slow (seconds) and CPU intensive to render. So you really don't want to have AWS or Google restart your application on AppEngine whenever they feel like it.

[scrollaway]

There's a few reasons why this scenario wouldn't be a good fit for cloud functions, but that "couple of seconds start-up time" can be almost entirely removed from the equation by keeping the Django instance alive (all cloud function type offerings will have a concept of cold and warm starts, and some way to control persistence across calls on the same "instance").

I've run Django on AWS Lambda in a a scenario that scaled between 25-250 calls per second depending on time of day (for a runtime of 5-30 sec). Moving Django's bootstrapping so it would stay warm across calls was very easy.

[stuaxo]

Silly question, but under nginx or apache do django instances persist or are they recreated for every new request?

[aeyes]

The standard gunicorn configuration (and the one shown in the blog post) never restarts worker processes.

gunicorn has an option --max-requests to restart every X requests but unless you have unfixable memory leaks there is no reason to do this.

Nginx can't directly run WSGI applications, you can do it with Nginx Unit which also never restarts processes.

[zbentley]

> unless you have unfixable memory leaks there is no reason to do this.

It's also useful to set this threshold to prevent long-lived connections to services/datastores not used by every request from accumulating and consuming resources on those services.

[ddorian43]

Now you just 5x their costs.

[latchkey]

Not if you do it right.

a) you get to fire the devops person, which saves $150k+ a year.

b) you add appropriate caching layers in front of everything.

c) you spend time adding features, which generate revenue.

I've done all of this before at scale. This whole case study was written about work I did [1]. Two devs, 3 months to release, first year was $80m gross revenue on $500/month cloud bills. Infinite scalability, zero devops.

[1] https://cloud.google.com/customers/gearlaunch

[Nextgrid]

> you get to fire the devops person, which saves $150k+ a year.

You are deluded or extremely short-sighted if you believe you can actually fire the devops guy. From my experience, the more you stray away from the conventional "dedicated server" paradigm the more you need a devops guy and you are in a very precarious position if you do fire him and something goes wrong.

[latchkey]

You don't hire the devops person until you've scaled to the point that you need one.

Additionally, your thought of having my company held hostage by a single devops person is terrifying. Now you need two of them, which is even more expensive.

It is a great way to bootstrap a company by saving on a salary (or two) that can honestly be engineered out for a lot of SASS businesses. It worked super well for us... and calling someone who did $80m in the first year deluded seems well, rude.

But, if you start off designing systems that scale on their own, you are much better prepared for when you do get some fast growth than dealing with hiring a good devops person (which is extremely hard, as they say.. all the good ones are taken).

At the end of the day, the actual elephant in the room is that django was the wrong choice. You end up having to go through a lot of contortions to make things work, as evidenced by the blog post. The architecture doesn't make things easy to spin up quickly... which creates a lot of bottlenecks. There are better cloud-based solutions.

[quickthrower2]

If you don’t have a devops person, then you end up with developers pitching in to fill that void. That’s OK and may be desirable but it is still a cost.