[m0zg]

Here's my recommendation (I've built several such machines for my own use):

1. Go with a 1600W PSU from EVGA or Corsair. Other brands are hit or miss if you ever need very high current on the rails. This will manifest in your machine suddenly powering off when all 4 GPUs are hit with data at once (as is typical at the start of an epoch)

2. Use a mobo with evenly spaced GPU slots, such as ASRock TRX40 Creator. That way you can install 4 GPUs eventually and use that 1600W PSU. You also get 10GbE for distributed training, which is nice.

3. Don't waste money on Titan RTX, get 2x2080ti's instead. Then after a while get two more. Buy blower cards which blow hot air _out_ of the case.

4. Use an extension cable to install SSD and do not install it under a GPU - it'll die eventually due to overheating.

5. Air cooling is fine

6. If you have more than 2 GPUs learn how to adjust fan speeds on GPUs. Crank them to 85-100% while training to prevent throttling.