[suprjami]

Actually budget friendly is RTX 3060 12Gb.

With one you can run 9B/12B models which are fine for text tasks like chatting or summarisation. Not for precision like tool calling or code.

With two of them you can run models up to Qwen 27B and 35B with a few-turn context window (8k-16k). Dense at 14t/s and MoE at 68t/s.

With three of them you can run 128k context, though you'll need a large format case and the right motherboard or PCIe riser.

I'm running three and even with a new case this setup cost me less than one 3090.

[VladVladikoff]

This seems quite unlikely. What motherboard are you getting three 16x GPUs on? That alone with the associated sever processor would be more than a used 3090, before even buying the three 3060s. Give full BOM and costs.

[suprjami]

I already had the PC. I just mean the extra purchase of the graphics cards.

The motherboard is an MSI Pro Z690-A.

The slots are physical x16. Electronically they are x16, x4, x1 which doesn't harm anything at all.