[sroussey]

How much can you quantize that down to run on a Mac Studio with 192GB? Is it possible? Feels like it would have to be 2bit…

[Davidzheng]

Less than 2bit i think. There's this IQ2 quant that fits