[mikece]

Does anyone do a measure of how long it would take such a workstation to pay for itself (including some nominal amount of operational cost for electricity) compared to simply doing ML on AWS/Azure/GCP? Seems like such a metric could be a useful measure for comparing such machines.

[CoolGuySteve]

A comparable workstation costs about a month of on-demand EC2 time or 3 months of spot instance time.

AWS GPU instances are really expensive.

The most cost effective imo is to build a workstation for development and then deploy to AWS spot if you need a cluster.

If you can't use a workstation for whatever reason, then use the new AWS feature to "stop" spot instances and use the spot instance as your workstation while being conscious of the high hourly cost and shutting it down when you're not working.

[FridgeSeal]

Azure ML/GPU instanced are also really expensive.

I did the maths recently and figured out I could put together a machine with a couple of 2080 Ti’s and have it pay for itself in a couple of months.

I’m very seriously considering doing it, especially as I’m the only data scientist, if I had a team I’d be more in favour of going to the effort of setting up cloud-based training jobs etc

[CoolGuySteve]

That's what my partners and I did. But we bought refurbished 1080 cards for about $300 each and Ryzen 9 hardware.

We're waiting for the 3000 series to come out which should be a large performance/dollar improvement over the current gen cards due to the smaller transistor size.