[thewataccount]

Looking at the comments I would double check the benchmarks because maybe the CPUs are faster then I thought for LLMs?

I know my 4090 for Stable Diffusion isn't even comparable to my i7 8700k and AFAIK the AMD/Intel offerings still don't compare for LLMs but admittedly it's possibly they've caught up?

I don't have a ton of time at the moment to keep looking, I have a very hard time believing the M1 can keep up with a 4090 at all, I just don't want you to drop 1.7k if I'm wrong :P

EDIT: Oh to clarify - The 4090 can definitely run the 30B model without issue with 4bit quantization.