[hobls]

Anyone using Lambda should _absolutely_ do load testing with different memory configurations. You will get different results, and should analyze what is best for your application.

When calculating the overall cost of managing your own instances you should also include time spent by your engineering team. There are particular tipping points in terms of overall requests per second at which point you'd save money by moving from Lambda to something like Fargate, and then even farther above that, you're better off using EC2. And then even above that, you should be running your own instances in a colo space. (And then at some point you should probably be building your own datacenters, and then at some point you should start colonizing the moon, and then... you get the idea.)

[chx]

> And then even above that, you should be running your own instances in a colo space

Why are people jumping from EC2 to colo and skipping dedicated servers? Mystery of my life. We were running the 75th largest site in the US some years ago (as measured by Quantcast), ran the numbers and colo was ridiculously expensive and way more troublesome.

[mmt]

> colo was ridiculously expensive

That's slightly surprising to me, unless you were limiting yourself to a particular geographic market for colo that was unusually supply-constrained at the time. (The SFBA during certain years comes to mind).

Sometimes rented servers, for a specific use case (or if the provider is overstocked on a particular model) are a great deal, but I've never seen that at the high-performance end of the spectrum, if they even offer such a model in the first place.

For the average and middle-performance cases, though, for truly comparable servers and connectivity (internal and external, which can be tough to find in the first place), I found rented servers to be moderately more expensive than colo plus buying hardware amortized over 3 years [1].

> and way more troublesome

This one, is the mystery of my life. You mention "magic smoke" downthread, but I've only experienced that once in my entire career and that was with proprietary hardware 2 decades ago.

Conversely, my experience with rented servers is that when there is a hardware problem, other than obvious failure of a replaceable part, "troublesome" doesn't begin to describe it.

[1] yes, including all the costs like installation/rack/stack, network ports, spares, etc. They're not de minimis, but it's maybe a few extra percentage points on the overall cost.

[hobls]

Oh, I was defining "EC2" as everything from small shared virtual boxes to EC2 bare metal. I suppose on-site dedicated servers is definitely a point I left out of my hypothetical list though! I used to work at an unnamed company that had a server rack plugged in between the kitchen and the ping pong table. Though I never witnessed such a thing, I'd be shocked if there was never an outage due to a particularly enthusiastic game.

[chx]

Not on site, no, renting them from some big provider. The advantages are huge: if the magic smoke gets released getting a replacement on line is Someone Else's (TM) problem. If you find you need to grow, you release the one you rented into the pool and the provider will be able to rent it to the next schmuck. EC2 bare metal is very expensive. In general, I am very cloud skeptic. I found the bare metal being a much better price/value for almost all websites. People are writing these complex apps that scale across database clusters and I am like... YAGNI. One database server (or a two for a hot spare) and maybe separate few web frontends is almost always enough.

[hobls]

I’m sure that makes plenty of sense for many applications! I’m very fond of some of the benefits of cloud infrastructure, but I’m sure you’ve heard the pitch.

[philipodonnell]

Are there any tools that allow you to load test Lambda services with different memory configurations?

[hobls]

Not to my knowledge. I'd just do something simple by changing your CloudFormation (or Terraform or whatever) templates, deploying the new config, and then running another load test. Could also just spin multiple different versions of the Lambda up, but at some point you need to start getting creative to actually generate enough traffic and running multiple load tests at the same time becomes a pain.

When it comes to load testing tools I like Vegeta[1], personally. (Though I've also used some much more complicated proprietary tools when testing at great scale.)

1: https://github.com/tsenart/vegeta

[reilly3000]

In Serverless Framework you can specify memory per function so it stands to reason making multiple copies at different memory levels would be pretty easy to test, even concurrently.

[hobls]

The real fun starts when you need to start spinning up a distributed load testing fleet to actually generate enough load. :P

[zlynx]

Has anyone done research on cooling datacenters on the Moon?