[hospitalJail]

Maybe someone can help me understand why people are investing into this.

Inhousing typically means falling behind in technology but having lower operating costs. That makes the company win, not the users.

If you hinge your career on Apple, they might make your technology obsolete on a dime.

Its not the fastest, its not the best, its not the cheapest, its not some combination either.

> 'compute per watt'

With AI? The local LLM models are near useless already. There will be a time to cut down on power, but from what I've read, there is currently ~no value even with a 4090 with 512 RAM.

I suggest avoiding Windows/M$, I am annoyed with Linux bugs, and google cannot be trusted. But all of that could be said about Apple as well.

I just don't see a future with Apple hardware, it gives me some serious Nintendo vibes where they are going to be some quirky niche that is just enough for marketers to sell it. Compute per watt seems like a wiimote that no one asked for, but suddenly claim is ultra important.

Maybe someone can change my view. I don't see who buys this when they are educated on the possible options.

[brucethemoose2]

> Maybe someone can help me understand why people are investing into this.

Buying a Mac for running LLMs is kinda like buying a Mac for gaming. Its thoeretically interesting, but I don't think thats a serious driver of Mac sales.

But:

- Finetuned local LLMs are good for specific niches, like roleplaying, text games, and helper bots for your own pile of data. And they are getting better at other niches like code completion for specific languages, or summarization.

- Remember that a huge selling point for Macs is iPhone/iPad development. The market for AI App Store apps is not small.This is also a reason to believe there will be some stability with the ML support.

[MuffinFlavored]

> - Finetuned local LLMs are good for specific niches, like roleplaying, text games, and helper bots for your own pile of data.

I can't see how they don't hallucinate/are leagues away from GPT-3.5 let alone GPT-4 level of quality of output. Am I mistaken?

[brucethemoose2]

They are better than GPT 3.5 (which I am generally not impressed with), but not as good as GPT4.

Again, the specialized variants perform very well in their niches.

[astrange]

Hallucinations are exactly what you want in a gaming model. That's another way of saying "creativity".

[gmerc]

You seem to assume hallucinations are a fatal flaw. You give it a document to summarize and see how often it hallucinates. Very little. Human performance.

Now often does a human make random shit up about general knowledge questions?

[Me1000]

There are a lot of ML applications outside of LLMs. Why would a developer invest in it? Because there are hundreds of millions of iOS devices out there where computer vision, text recognition, etc would be useful features.

[Geee]

Desktop computers are heat-limited. We could have much faster computers if we found a way to cool them down. Thus, compute per watt is the ultimate metric to optimize for. If your cooling capacity is 500W, then obviously you'll want to fit as much compute in that as possible.

Mobile devices are energy-limited. You'll want to do as much compute as possible on a limited battery.

[IOT_Apprentice]

My question to you is what are you currently using as an alternative for the COU/SOC in your personal & work environments?

Intel? AMD Ryzen?

Apple has taken their ARM approach and scaled it to all their platforms.

Amazon now is on what, Gen 2 or 3 for their graviton platform in AWS.

And what OS are you using if you don’t trust Microsoft, Linux or Apple?

[brucethemoose2]

CPU arch isnt't even that critical here, as Apple is talking about Metal.