[jllyhill]

To be honest, I'm getting tired of a "laptop" in every one of these clickbait titles turning out to be $3000 Macbook. Sure, it's impressive to achieve this degree of the LLM compression, but I really don't like that the title implies local LLM becomes a viable for an average person with the actual hardware being out of reach for 99%.

[kelnos]

I enjoy them because I have a reasonably beefy laptop (non-Mac, though, so I can't try out this particular project), and it's nice to see what you can do with laptops in this space.

I think, though, maybe consider your biases? Whenever I see a headline like this, I absolutely don't assume it's going to run on any random laptop. I don't even expect it to run well, or at all, on my laptop (for stuff that isn't Mac-only, that is), which has an iGPU. I'm generally a big LLM/AI skeptic, but I find your brand of cynicism/dismissal to be kinda boring and uninteresting.

[freehorse]

You can probably go lower than $3000, I expect, ime, an M1 max with 64GB ram to have similar performance, and you can find such one used with less than $2000 or so probably.

In any case, I do not think that the range of people that have an M1/2/3/4 max macbook is that narrow, eg people who may do video editing or who benefit from having one of the fastest multicore laptops. It is handy to be able to do work with a machine you already may own for separate reasons, though it is definitely more to the side of a "pro device" than "basic consumer device".

[kroaton]

It could just as easily be a $3000-4000 Strix Halo laptop.

[Computer0]

Yeah I understand the sentiment, I think it should’ve been “,on a laptop!” instead of “on a laptop”

[throw284959]

I ran full version of this model without any swapping on cluster of 2x $3000 laptops (strix halo zbook 128GB) at about 20 tokens per second.

I would say it is in reach for normal person. If anything buying it was great investment, it is work tool, I will probably sell it for more than what I bought it for :)

[prmoustache]

> I would say it is in reach for normal person.

That is a very wealthy country centric thing to say.

[mycall]

..and to think $2=3k is basically a minimum to really use local LLMs effectively, with prices definitely rising globally. Most people will just stick with online services for LLMs.