[nickthegreek]

I can run the 30b 4bit model on my m2 air that has 24gb of ram.

[jpp4e]

Hi nickthegreek!

Could you tell me how you did that? Did you use FastChat or something else? Which model to download? What command to run?

Thank you!!!

[nickthegreek]

https://huggingface.co/Pi3141/alpaca-lora-30B-ggml

I believe I’m using alpaca.cpp with a command:

./chat -m <bin filename>