[Dylan16807]

> The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops.

I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about.

Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing.

> Imagine leasing out idle time on your desktop or even laptop for cash.

https://vast.ai/

[b112]

You're missing the point. The issue is "how is the thing used" not trying to make identical break points in differing tech. A self driving car cannot have object detection, collision avoidance, human detection offloaded to the cloud. Ever. At all. A network drop can't mean it smashes into things. Or stops unsafely.

The same is true with an android. Imagine it turning on an frying pan, cooking dinner, and then going offline part way through. Or turning on a tap to wash something, and going offline while the sink overflows and destroys the house.

There are myriad of such scenarios, but local compute is absolutely, 100% necessary. Anyone betting the farm on putting network controlled devices into homes for any serious task is going to lose their shirt. Local compute is an absolute requirement, and a few TB of RAM and local compute will be nothing over the scope of this discussion (a few years minimum, just to build and kick off all these new fabs).

By the time these fabs are online, expect most smart phones to have 1TB of RAM and significant llm capable compute (gpu or other custom silicon). I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. Note I'm saying flagship, there will be of course economy models as always. Certainly laptops will be sold in multi-TB RAM configs.

[Dylan16807]

I'm not missing the point. I'm saying that some amount of local compute is necessary but the really big stuff can be remote.

You don't need terabytes to turn things back off.

Not that I want to trust "not setting the house on fire" and "not flooding the house" to this kind of model in the first place...

> I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM.

I'll be astonished if they have 50GB.

Have you looked at RAM size/price trends? I'm not even talking about the last year, just the pattern before that. We're not in the 80s and 90s anymore. The most recent price lows were roughly $3.50/GB in 2013, $2.50/GB in 2016, and $1.50/GB in 2023. If we're lucky the cheapest stuff will hit $1/GB in a few years, and the kind that would actually fit on a phone motherboard would be significantly more than that.

Samsung hit 16GB on their top model in 2020 and it's either been 16GB or 12GB ever since. Apple only went up to 12GB in the last year. Google offers 16GB. A couple niche offerings have 24GB. Why would these RAM numbers even double during the next four years?

[b112]

Well there's only one reason, so you do indeed know why.

Here's the thing, by 2028, maybe 2029, 1/2 the cloud AI providers are going to go bankrupt, or severely downsize. This will likely be due to massive reduced power and processor requirements for AI. GPUs are a stopgap, and they won't be used much longer. When that happens, and the number of servers drops by 90%, along with similar power requirements, a lot of people are going to go tits up due to massive, empty datacentres and compute.

That will absolutely flood the market with cheap obsolete GPU + RAM, used servers, you name it. Current RAM prices will plummet, and those able to withstand the storm will have to compete again, not just ride the wave like they've been doing now.

RAM has been ridiculously high for a long time, it won't be by 2030.

Couple that with local SoC with dedicated compute for LLMs, and you have the perfect storm for local, on phone compute. Why this? Well, because LLMs will be in everything! Certainly in robots, autonomous weapons, and anything the crazies can shove it into. Phones won't be the unique case, it'll be sharing a hardware market with robots, and 100 other things using the same SoCs.

Local compute will be easy... RAM makes a massive difference, huge as you know, difference in local compute. And with the datacentre market in tatters, and people becoming more and more concerned about privacy, people will likely not want to whisper to a cloud LLM, about their fetishes for donkeys or moose or whatever turns their crank.

Not to mention, other aspects of their lives.

In as RAM will be super cheap, that's where I see this going.

I have a sneaking suspicion you don't agree with this assessment. I guess we'll come back here 2030 and see what's what.

[Dylan16807]

What do you think the production cost of RAM is? Even if we perfectly fixed competition and ended the AI bubble right now, I don't see how new RAM prices would drop below half the previous low point by 2030. Those fabs are legitimately ultra expensive, that's why almost all the RAM makers went out of business and we're left with the current cartel.

Also "massively reduced requirements for AI, so much so that we have 90% fewer servers despite the Jevons paradox" is not compatible with "high quality models make effective use of terabytes of memory" and "LLMs will be in everything".

And even if you could perfectly reuse all the memory out of the collapsing datacenters, that's not enough to make many of the phones and laptops you're describing. We'd need a 10x increase in manufacturing on top of the massive price drops, and even if we started building those fabs today they wouldn't be done in time (and we wouldn't have enough equipment to fill them).