[rmunn]

> I reject all notions that these are mutually-exclusive concepts.

Agreed; just because I "rarely" pull out the calculator doesn't mean I never use it. Sometimes my wife asks me "If this cake recipe calls for filling a 9-inch pan with batter, and makes 3 layers, but I have an 8.5-inch pan and I'm making two layers, then how much should I reduce the recipe?" The π terms cancel each other out in that math, but I still can't do (4.25×4.25×2) / (4.5×4.5×3) in my head faster than I can punch it into the calculator. (It comes out to about 0.59, BTW, meaning she should use either half or two-thirds of the recipe depending on if she wants a slightly thinner or thicker cake).

Bringing this back to the broader LLM-related discussion, the lesson I took away from the Fable kerfluffle was "the model will usually, but not always, be available." You might have an Internet connection hiccup, you might have hit your subscription usage credits for the week and not be willing to pay API pricing, the model might even be taken away from you. (Most of which are problems you won't face if you run a local model, but that's a different discussion). So just as it's important to learn mental math even when you have a calculator as backup, it's important to retain the skill to solve many categories of problems without the LLM, even if you still reach for the LLM for the categories of problems that it can solve faster than you can.

[ssl-3]

Sure. So to bring it all back together:

I don't think I'd want to rely upon having a rental pocket calculator to use. It could become unavailable at any time, for any reason. For something as common, inexpensive, and easy to find as pocket calculators have been for decades, renting one sounds crazy.

But that's the way that clown-based LLMs are: They're rented. They could disappear at any time.

And sure, I do use the bot to get some stuff done, anyway. I think of it as somewhat akin of the time-sharing computer systems of yore, where people absolutely rented (often far-away) computing resources to accomplish their work.

Those time-sharing systems were only temporary. The giants like Control Data Corp fell out of favor as computers became smaller, faster, and cheaper. (And then they came back in the shape of things like AWS, but that has a lot of non-technical things driving it; AWS is, strictly-speaking, completely optional.)

Unless we stop making computers smaller/faster/cheaper (which we haven't, despite the present-day supply/demand market blip) I think it is certain that we will get there again with LLMs, where the giants fall. Not this year, or next, but small systems will catch up.

And these little machines don't even have to win the race. They just have to be Good Enough.

So it seems inevitable that people like you and I will live to see fantastic local LLMs happening with affordable hardware that fits on a desk.

And then, eventually, into our pockets -- like a calculator.

[rmunn]

Given the power draw of the GPUs and RAM needed to run those local LLMs, I don't see them being comfortable to hold in one's pocket anytime soon. :-) Not until some not-yet-imagined breakthrough in heat dissipation is made. But a thin client that talks to the local LLM on your private network, that's already possible today. So yes, within the next 5-10 years I fully expect I'll actually want to use an AI assistant on my phone. Currently I go through the settings and turn everything related to Google Assistant off, and do that again after every Android update. But once it's talking to a model that I control, instead of a model controlled by the world's largest advertising company, I'll feel the opposite way about it.

P.S. If "clown-based LLMs" was an autocorrect-assisted typo for cloud-based, it was inspired. :-) If you typed that on purpose, it was also inspired. Have an upvote.

[ssl-3]

Power consumption per unit of work keeps going down, too. Our pocket supercomputers are astoundingly efficient compared to what it used to take to get the same work done at relatable points in the past. We'll get there.

Before we get there, we'll have network connectivity to our desktop AI boxes at home. That's pretty good, too.

And if this clown-bot[1] boom is a bubble (as I believe it is), then it's just a matter of time before it pops. The blast radius is unknown, but at one end it seems likely to result in something between potentially-idle fabs that have already been mostly paid for. Idle factories are bad and it makes sense to avoid that even if it is expensive, so this means cheap hardware.

At the other end, it means failed companies with fabs that are sold for pennies on the dollar alongside a resolute unwillingness amongst the investors (who just had their asses handed to them) to start on Boom 2.0. This latter scenario would mean golden age of cheap hardware.

I think we'll be fine.

[1]: Yeah, clown is on purpose. It fits as a replacement in any technical parlance where "cloud" would be used. :)

[rmunn]

I'm of the same opinion re: the bubble. The dotcom bubble held on longer than I thought it would (I had been predicting that it would burst in 1999), so I'm not sure I should be making predictions as to when the rental-LLM bubble will burst. But I'm sticking with the hardware I have until memory prices finally fall again. Whether that's only because of spare fab capacity as you mentioned, or because one of the major companies has gone bankrupt and their already-purchased hardware is being sold off at fire-sale prices (perhaps to other companies, who then reduce their new-hardware orders accordingly, which again results in spare fab capacity), I'm going to buy more RAM when it's cheap.

... And I've just been agreeing with you on all points with this comment, haven't I? Time to wrap up the discussion, then: once total agreement is reached then there's not much point in continuing the discussion unless someone has something new to say, and I don't have anything new to say on this topic. See you around.