[seanmcdirmid]

A max with 64 GB of ram should be able to run this (I hope). I have to wait until an MLX model is available to really evaluate its speed, though.

[pickettd]

Looks like MLX quants are released:

- 4bit: https://huggingface.co/mlx-community/QwQ-32B-4bit

- 6bit: https://huggingface.co/mlx-community/QwQ-32B-6bit

[seanmcdirmid]

I downloaded the 8 bit quant last night but haven't had a chance to play with it yet.

[mettamage]

Yep, it does that. I have 64 GB and was actually running 40 GB of other stuff.