[fbdab103]

I do not purchase hardware, but $1MM is way above what I would have expected. Going to Dell, the most expensive pre-built rack mount starts at ~$30k. Assuming 16 of those only gets you to $480k. Throw in an extra premium for the rack itself + small company margins still leaves me reaching to get to that price point.

[toomuchtodo]

Price delta is out of the box cloud orchestration value (imho). Most large enterprises would struggle to build this themselves (Mesos->OpenShift->Kubernetes/Tanzu/etc), so you’re paying for turnkey cloud on prem. Probably save in the long run considering public cloud margins.

Enterprise CIO doesn’t want a hobby project (attempting to cobble together internal cloud orchestration and infra), they want to be able to show immediate business value. You charge what the market will bear. I’ve seen many companies with thousands of employees and spending millions, even tens of millions a month, on public cloud providers and just flail, unable to get to steady state post transformation (even after years of trying). This is made for those folks, especially with Broadcom having VMware self inflict harm on itself with recent strategy decisions.

“Write check. Cloud up.”

(no affiliation)

[mise_en_place]

I mean…you could also just get a z/VM system and have a few LPARs on it and just use Ansible for orchestration. Why wouldn’t an enterprise CIO just go for a mainframe system?

[dns_snek]

"just" is doing a lot of heavy lifting here, I'm not the target customer base for one of these, but if they can deliver a server rack that teams can plug in, turn on, and start deploying workloads to it in the same way they currently deploy to public clouds with familiar tooling, that seems extremely valuable to me.

It's going to depend on how well they manage to pull off the magic trick of "little or no configuration and maintenance required". If things start breaking in hard to diagnose ways, it's going to be just another broken appliance that requires expensive maintenance, and companies will be questioning why they didn't DIY it in the first place.

[panick21_]

If there is one company that has made 'make it easy to debugging issues' their core philosophy, its them.

Its almost all open software, that helps a lot. They add a minimal amount firmware, rather then the many, many million lines of firmware that is usually around. And most of the stuff they added is Rust on a micro-kernel. (Check out the talk I linked top level to see some of their low-level debugging infrastructure).

To bad they can't (yet) get open firmware in the NIC, the SSDs and some of those other places (Time for an Oxide like company that makes P4 driven NICs). But nobody else can really offer that either.

The only real issue for them is that Illumos is the host OS. Its open source and stable of course, and has good debugging tools. But in terms of industry experience, the amount of people with deep knowlage of the system are harder to find compared to Linux.

The of course also add some complex software on top that will have to work properly, moving VMs, distributed storage and so on.

Full DIY is pretty damn hard, you need a serious team to pull that off. The Dell VxRail/VMWare is the more reasonable competition. I think VMWare going full Broadcom mode will make them more interesting. Buying into that ecosystem isn't that appealing right now.

[panick21_]

Getting the same performance and feature out a mainframe will be considerably expensive I would guess. And in addition to that you are buying into an incredibly closed ecosystem where prices only go up from there.

You are also paying for a bunch of stuff you don't need. Most people just don't need to hot swap a CPU or turn these single socket 128 core machines into a gigantic 4096 machine either.

Simply moving virtual machine off and restarting or replacing a sled is enough for the waste majority of use-cases.

This is still pretty much commodity single socket server platforms, just with more sane and open firmware and a sane open source software stack.

[bnprks]

Are you sure you're comparing equivalent memory and storage specs? I needed to go into the customization menus in the Dell configurator to spec something equivalent, where prices started going up quite rapidly.

For example "3.2TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier" is $3,301.65 each, and you'd need 10 of those to match the Oxide storage spec -- already above the $30k total price you quoted. Similarly, "128GB LRDIMM, 3200MT/s, Quad Rank" was $3,384.79 each, and you'd need 8 of those to reach the 1TiB of memory per server Oxide provides.

With just the RAM and SSD cost quoted by Dell, I get to $60k per server (x16 = $960k), which isn't counting CPU, power, or networking.

I agree these costs are way way way higher than what I'd expect for consumer RAM or SSD, but I think if Oxide is charging in line with Dell they should be asking at least $1MM for that hardware. (At least compared to Dell's list prices -- I don't purchase enterprise hardware either so I don't know how much discounting is typical)

Edit: the specific Dell server model I was working off of for configuration was called "PowerEdge R6515 Rack Server", since it was one of the few I found that allowed selecting the exact same AMD EPYC CPU model that Oxide uses [1]

[1]: https://www.dell.com/en-us/shop/dell-poweredge-servers/power...

[amluto]

> For example "3.2TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier" is $3,301.65 each

That’s the pricing for people who don’t know to ask for real pricing — it’s an absolute joke. I don’t know how might extra margin gets captured here, but it’s a lot.

Even in teeny tiny volumes, Dell will give something closer to real pricing, and a decent heuristic is that it’s at least 2x cheaper.

This is a real SSD. Dell likely buys this brand and others:

https://www.serversupply.com/SSD/NVMe/30.72TB/KIOXIA/KCMY1RU...

Yes, that is almost an order of magnitude cheaper per TiB. If you buy from a sketchier vendor, you’ll get all the way to 10x :)

[1123581321]

Doesn’t Oxide also negotiate pricing?

[Temporary_31337]

And they keep the margin to have money for R&D. I kind of get it because it’s low volume for now but I don’t necessarily see the appeal of being an early adopter here.

[panick21_]

The appeal is that your a buying into an open infrastructure system, rather then closed firmware and VMWare (now owned by ... Broadcom).

[bnprks]

Thanks for the link, and very good to know. I've always struggled to find component prices for Kioxia drives and higher-capacity RAM sticks so it's good to see I can finally look these up on serversupply when I'm curious.

[dilyevsky]

$480k + switches + management + support + virtualization licenses + integration - it adds up. It will also probably take you at least 3x as long. I can think of lots of examples where this premium for apple-like ux is totally worth it