Batch inference is much more efficient. Using the hardware round the clock is much more efficient. Cloud can absolutely pay more for hardware and still make money off you.
Cloud can pay more for RAM until all the RAM producers withdraw from the consumer market, then prices will go back down.
End users will still get access to RAM. The cloud terminal they purchase from Apple, Google, Samsung, or HP will have all the RAM it will ever need directly soldered onto it.
Doesn’t Apple place RAM directly into the SoC package? We aren’t even talking about soldering it to mother boards anymore, it is coming in with the CPU like it would as a GPU.
The next step, I think, will be a "cash for clunkers" program to permit people to trade in old computer hardware to the government—especially since operating systems that do not collect KYC data on their users will soon be illegal to operate.
The hourly cost problem is worse for agents than single-model calls because context accumulates across steps. each tool result re-bills everything before it. Rate limits are a ceiling but the quadratic curve hits you before the ceiling does. We built Traeco to surface that curve at config time, not billing time. traeco.dev