[kiratp]

We should be happy that compute is once again improving and machines are getting outdated rapidly. Which is better - a world where your laptop is competitive for 5+ years but everything stays the same? Or one where entire new realms of advancement open up every 18 months?

It’s a no contest option 2 for me.

Just use llama.cpp with any of the available UIs. It will be usable with 4 but quantization on CPU. You can use any of the “Q4_M” “GGUF” models that TheBloke puts out on Huggingface.

https://github.com/ggerganov/llama.cpp

UI projects in description.

https://huggingface.co/TheBloke

A closed source option is LMStudio.

https://lmstudio.ai/

[elicksaur]

“New realms of advancement” could open up because of faster computation algorithms. Those hypothetical scenarios don’t have to be mutually exclusive.

[seanvelasco]

i love this perspective! makes me really happy of the advancements going around, and not feel sad about my macbook m1 getting old