[YetAnotherNick]

> AMD MI300x

Which costs significantly more than H100 at least when renting[1]. Also the price of hardware isn't significantly lower.

Also, both AMD and Nvidia have been deliberately stopping progress in cheaper consumer graphics card by not increasing VRAM and removing things like fast interconnect.

[1]: https://getdeploying.com/runpod

[latchkey]

Your source is a paid advertisement[0] that only lists providers who pay the tax. I'm not really interested in that sort of game. We've done the research and our MI300x pricing is competitive with H100's and even more so if you consider the amount of vram in a MI300x.

I am also not sure if it is deliberate or just realizing that running this stuff is error prone and requires a lot of capex, power and infrastructure. It is difficult to support that at the consumer level, so why bother when enterprises are now offering super computers for rent. It is not their wheelhouse, so I can see why they do not want to take on the extra risk.

[0] https://getdeploying.com/plans

[amzans]

Hey, I'm the guy behind GetDeploying. To be clear:

This is a side project and the vast majority of companies were added by me without being paid for it. I now charge to get listed or add a banner because the site takes too much time for me to maintain.

And totally agree, AMD GPUs are not covered enough. Happy to list your company at no cost to help me fix that. Feel free to email me if interested.

[latchkey]

Hey, thanks for the response. I'm currently not interested in being listed on a paid site like this, even if the offer is free. I don't think that is what the community needs.

What good is any of this other than driving clicks for your benefit? If I'm going to get any traffic from your site, it is all going to be driven by people just searching or quoting comparisons, not actual sales.

For example, right now, you list another MI300x provider. Right at the top of the page you parrot their bogus claims about 20k GPUs by 2024. They don't have pricing, it is just "contact us". "Based on our records, XXX has at least 2 data center locations around the world"... yet it lists both of them in the US, not "around the world". I could go on and on, but what I know is that I don't want to be associated with something like this.

Sorry for the truth bomb, but if it is taking too much time for you to maintain, you should shut it down or find someone else willing to maintain it properly. Having incomplete and bogus data isn't helpful for anyone.

[amzans]

Thanks for the feedback. There's certainly room for improvement.

[latchkey]

I saw your original reply, good editing.

[YetAnotherNick]

> We've done the research and our MI300x pricing is competitive

More info would be appreciated. Because I tried finding the pricing for all the providers and they aren't similar. In my research, in almost all the cases, 2*A100 is superior than both H100 and MI300x in VRAM, performance and pricing if the usecase supports multi GPU.

[latchkey]

If the A100 pricing you've found works better for your use case, then go for it. I'm not here to convince you into something you don't need or want.

Please bear with me though, I would like to take this opportunity to explain a bit how this industry works cause I feel like there is a lot of justified confusion.

You'll find that there is no public pricing because it is usecase dependent. Everyone needs something unique and per/gpu/hr pricing doesn't really quantify the entire hardware stack. Inference doesn't need machines with 8x400G networking. One person needs a week, others need multiple years. Some people want CFD, others want HFT. Frankly, there is also a supply/demand aspect... not many companies offer or have MI300x for rent and we've taken on that capex risk for you.

That said, I can speak about what we are doing and where we are going that aligns with our overall transparency. We've got base weekly pricing now in public (which is competitive to H100's) and we're working on publishing a set of public % discount tiers that should cover longer term rentals. Eventually, we plan to offer inference specific hardware, for even lower prices, since it has different requirements that do not cost as much. We're also going to be offering an hourly docker experience soon too.

At the end of the day though, we're not trying to be the cheapest. We will let others fight that race to zero. We're trying to be the best in our own niche. That happens by picking the best data centers, best hardware vendors, professional next business day support contracts with Dell, and white glove customer support. This sets us apart and above the rest.

Those are areas that the capex moat, is very difficult to compete with. You'll try the cheapest route first and realize that when you see things overheating or failing and taking forever to resolve, you will wish you had come to us. The idea is that we've spent quite a bit more to de-risk your business, as well as ours.