[narrator]

Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?

[magic_hamster]

Macs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.

[adam_arthur]

You can do LLM inference without CUDA just fine. Download Ollama and see for yourself

[doublepg23]

I'm assuming this won't support CUDA either?

[egorfine]

For LLM inference - yes absolutely.