[2024throwaway]

Running with a Local LLM on a Mac M1, this completely locked up my system for minutes. I tried to let it run, because the progress bar was ticking every now and then, but after 10 minutes I gave up and killed it.

[raxxorraxor]

You are running the wrong model. Get a smaller one Macs are usually ok with their performance since the GPU has access to main memory. A decent Nvidia GPU with lots of memory is still king though.

I run codelama:7b on a MacBook air, and even auto complete is partially usable.

[tarruda]

I wouldn't recommend unless you got at least 16gb ram (though possibly more is needed depending on what model is used).

[2024throwaway]

I do have 16gb ram.