"All it takes to run" might be fair if you paid $2400, but right now the total price is way closer to $10k (almost 5k for the RAM and 2k each for the GPUs). Today that is a lot of expensive hardware.
Agree'd here. I am seeing a pattern in this thread - Everyone here seems to justify paying substantially more money to self-host hardware thinking they are saving in the medium-long term. Frankly from the perspective of the company I'm with - lost opportunity cost is more important than trying to future proof on a hunch that AI vendors are going to pull the board out under you. There's alot of competition in this space and we're seeing these frontier models continue to get better for the foreseeable future. As such, you're dumping alot of money upfront on hardware which is not guaranteed to be relevant in a few years time. The benefits to me seem pretty speculative, minus the point of security.
512gb 2400mhz ddr4 ram = $1600 not $5000. https://www.ebay.com/itm/188284985172
You can get creative and source 2-3 2080ti 22gb from China for about $250 a piece. You can either be resourceful and find a way or find a whole bunch of excuses.
> You can either be resourceful and find a way or find a whole bunch of excuses.
How about addressing this false dichotomy with the likelihood that someone who is new or interested in a tech isn't willing to drop thousands of dollars on used hardware for a whim or learning exercise.