[agnokapathetic]

> My friends at Replicate told me that a simple rule of thumb for A100 cloud costs is $1/hour.

AWS charges $32/hr for an 8xA100s (p4d.24xlarge) which comes out to $4/hour/gpu. Yes you can get lower pricing with a 3 year reservation but thats not what this question is asking.

You also need 256 nodes to be colocated on the same fabric -- which AWS will do for you but only if you reserve for years.

[thewataccount]

AWS certainly isn't the cheapest for this, did they mention using AWS? Lamdba Labs is 12$/hr for 8xA100's, and there's others relatively close to this price on demand, I assume you can get a better deal if you contact them for a large project.

Replicate themselves rent out GPU time so I assume they would definitely know as that's almost certainly the core of their business.

[sebzim4500]

Maybe they are using spot instances? $1/hr is about right for those.

[celestialcheese]

lambdalabs will let you do on-demand 8xa100 @ 80GB VRAM/GPU for $12/hr, or reserved @ $10.86/hr

8xA100 @ 40gb for $8/hr

Replicate friend isn't far off.

[pavelstoev]

model-depending, you can train on lesser (cheaper) GPUs but system-level optimizations are needed. Which is what we provide at centml.ai

[IanCal]

Lambda labs charges about 11-12/hr for 8xA100.

[robmsmt]

and is completely at capacity

[IanCal]

But reflects an upper bound at the cost of running a100s.