[patricktlo]

That's amazing, any chance of running it on my trusty GTX 1060 6gb, or that's not enough VRAM?

[yacine_]

From my mental math, you can run 4bit quant 7B, but the quality for that might not be what you're looking for.