[jcgrillo]

> The main place RAM usage is going to get optimized is on the server side

Agree, but it's going to be a long, hard, potentially ultimately unsuccessful uphill slog. I think the main obstacle is basically 2 decades of inertia. Approximately everyone believes that server side software needs to scale horizontally. So we optimize systems for this property--I can make it work better by just throwing more computers at it. This was a lot more important ca. 2004 (MapReduce) than it is ca. 2026. For resiliency and redundancy, how many nodes does your service actually need? If you're running more than that number, do you have some kind of justification as to why? This is the mindset we need to try to move towards, but it's very different from how we currently do it.

EDIT: Another good question to ask, which we don't do enough: "does it really need to be a separate service?" Using data in RAM that's already been allocated is probably better than allocating more and shipping data over the network...

[mike_hearn]

FWIW I work part time on projects that are all about reducing RAM usage server side! RAM has been a big expense in the cloud for a long time. AI makes the situation critical but there's a lot of work done already. Consider that functions-as-a-service are basically all about shutting down idle servers to reclaim their RAM.

[jcgrillo]

FaaS are really interesting, because the tradeoffs seem "jagged" (to use a word I learned recently). Like, IIUC this is more or less what's happening when a cloud function is invoked:

1. Some event triggers the function (HTTP request hits load balancer or whatever)

2. A VM is loaded from NVMe (possibly on another computer, and transferred over the network) into RAM

3. That VM boots up

4. The function code runs

5. The VM is terminated

Naively, this probably takes a lot more RAM (not to mention all the other computing resources) than to just have another HTTP handler sitting idle on some computer somewhere.

So this is only a reasonable choice if doing this intensive computation for a short amount of time, and using a large amount of resources for that short amount of time, is somehow better than doing a much smaller amount (maybe none at all) of computing over a longer amount of time. It's kind of hard to tell what the actual conditions are that merit one solution over the other. The other layer is the cloud pricing model, but that's a whole other can of worms..

I have yet to encounter a system that didn't have at least some computers (or cloud instances) that always need to be running. Those computers are almost always a bit overprovisioned (not using 100% of their CPU or RAM or IO all the time). So it has always seemed to me that instead of running a cloud function, it would be cheaper in terms of compute resources to execute the code from step 4 above on one of those machines.

Obviously we have cloud functions for a reason, people like them and find them useful. But is there ever a regime where they're optimal?

EDIT: I guess I've implicitly assumed the code in step 4 is doing a small amount of work. Maybe there's a goldilocks zone of work-per-invocation where this model is more optimal? But then of course there's always the standard executor pool model, and that way you don't have to spin up a VM per task invocation...

[computably]

> Naively, this probably takes a lot more RAM (not to mention all the other computing resources) than to just have another HTTP handler sitting idle on some computer somewhere.

CF Workers uses V8 isolates rather than (OS/HW-level) VMs.

Also, the RAM usage isn't really a differentiator between FaaS and any other cloud instance. If a VM is used for FaaS, it's almost certainly also used across the platform for multi-tenancy, invisible live migration, etc.