But for VM, do customers pay by CPU usage or runtime? I am sure the Netflix of this world optimise their CPU usage but I also suspect the majority of VMs are mostly idle or have little traffic as their task are either intermittent or are sized for peak usage.
Not on AWS but as far as I can tell from their pricing estimator[1], for 24/7 instances you pay based on time running, not cpu usage.
Can see the appeal of AWS for scaling, redundancy, cross region availability, etc., but for long lived services it seems you pay a high price compared to services like Digital Ocean where the per VM cost is significantly lower.
Technically, you pay for both time (hours on) and CPU Usage (instance tier). Its not like different instance tiers (at least in the same class) use fundamentally more or less powerful processors. They all use the same processors, you just get more or less of it depending on what you pay.
Conceptually it is "pay as you use" by CPU usage, but just rounded into buckets by instance tier.
Of course, there's a lot of underutilization within each bucket, because the granularity isn't per 1% used, but (more or less) per 100% used (aka each core). And also, most applications can't switch instance tiers easily to adapt to demand (though some certainly can).
>>> Its not like different instance tiers (at least in the same class) use fundamentally more or less powerful processors. They all use the same processors, you just get more or less of it depending on what you pay.
There is a variety of CPUs. You can "cat /proc/cpuinfo" to see what you got.
>>> most applications can't switch instance tiers easily to adapt to demand (though some certainly can).
Most applications can and do change. Changing the instance type is just a reboot of the machine.
Etc. The only exception I'm aware of are the burstable instances; AWS's documentation doesn't explicitly guarantee a CPU bin there, it just says "high frequency Xeon".
The difference between an m5.large and an m5.xlarge isn't that you're getting a "faster" processor (meaning higher gigahertz or newer architecture). You're just getting "more" of the same processor (more cores). This is different than on, say, GCP, where you just ask for cores and you can specify if you specifically want a broader generation of Xeon (like Broadwell), but you can't be guaranteed specific chips.
My intention behind saying that most applications can't switch instance tiers is more to point out that most applications aren't prepared to handle node failure, not that there is something intrinsic to the node types which stops them from being able to switch (that'd be much more rare).