[weiran]

You should be able to run Q3 and maybe even Q4 quants with 32GB. Even with the GPU as you can up the max RAM allocation with: 'sudo sysctl iogpu.wired_limit_mb=12345'