[h14h]

This feels like getting a foot in the door to ensure Apple doesn't entirely eat Nvidia's lunch if AI inference workloads start to shift from cloud to local.

With MLX, Apple is building an answer to CUDA, and if people start switching from ChatGPT & Claude to some app that runs on their M5, suddenly Apple starts to look like Nvidia's biggest competitor.

If Nvidia doesn't have a pathway towards getting hardware into the hands of consumers, it could be a really difficult road ahead for them.

[selicos]

Apple seems to still own the creative space. If those tools are able to run local models for any AI workflows suddenly anthropic/etc could lose a massive segment. Or at least demonstrate to others wanting a slice of the cloud AI profits it can be done.

I'm here for it. Local models can do a lot of what I need at almost no cost, plus the fun of making them work better or building a new system to handle that aspect of my home lab. A Strix Halo system may not be amazingly fast but at 128gb of RAM it can keep up with most open models worth exploring.

Based on June 1 Copilot Pro plan premium token burn and cost, unless you REALLY know how to use cloud AI efficiently and are tooled up to do so a local LLM on hardware you may already own is very appetizing.

I converted a lot of work today to a 6.5gb local LLM on a 12gb GPU and no, it's not as good. But it is 'free' or at least feels that way, especially when I need to redo something and my copilot premium request % doesn't change.