[gbolcer]

3060 12GB are the best things you can buy right now. They are cheap, have a ton of memory--which seems to be the issue w/ image generation--and you can fit four of them into the cheapest motherboards.

3060ti 8GB, 3090 24GB, and 4000 series all have performance benefits, but for now this one is off the charts.

[BatteryMountain]

With this card you can also run Open AI's Whisper with the Large model (the multilingual one!), as it requires 10GB.

[crabbycarrot]

Highly recommend quantizing the model (https://pytorch.org/tutorials/recipes/recipes/dynamic_quanti...). I converted the large model to use int8, and I'm able to run it 5x real-time on CPU with pretty low RAM requirements with still very good quality.

[Const-me]

My implementation of Whisper uses slightly over 4GB VRAM running their large multilingual model: https://github.com/Const-me/Whisper

[layoric]

Another one to consider is the A4000 16gb. I recently bought an ex-miner card for ~$500 usd . They are around a 3070 with a decent amount of memory for training scenarios, and are single slot cards. I believe there are a lot of these workstation ex miner cards which are pretty heavily discounted.

Combine this with a second hand X99 / 2011-v3 platform like the Dell Precision T7910 dual socket Xeons and you can have a pretty decent homelab for ML workloads. The Dell can come with a 1300 watt PSU and can fit 4 of those cards comfortably (5 with reduced PCIe lanes on one) since they are 150W each.

[lithiumii]

Also A2000 12GB, it's a slightly less powerful 3060, but it only requires 75 w of power, meaning you don't need to plug in a power cable.

[gzer0]

Any suggestions on which Motherboard would be ideal for a 4x 3060 12GB setup?

[janekm]

For image inference 12GB is ok for now (but you may not be able to use all future models given that T5 language model is becoming popular), for training I'd consider 24GB the bare minimum.

[hnrodey]

A few weeks ago I got a 3060 12GB for $250 from a guy on FB Marketplace.