[InsideOutSanta]

I just ordered my Framework without any memory or storage, hoping that by the time it arrives, I'll be able to pick up some RAM and an SSD for a more reasonable price. If not, I'll just grab something from a drawer and use it underspecced until prices normalize.

[pcblues]

Sensible thought. I very much hope there is a glut of one-three year old ram and GPUs on the market in about one year when the AI market "cools" and the ear-marked components return to the market.

The banks that lent the AI industry the money are already trying to sell their debt.

[chao-]

Unless you meant buying an entire server (instead of laptop/desktop components), it won't work out the way you are describing. Prices may come down, but the components for the datacenters market aren't fungible with the components for laptop or desktop. You might not know what is being "earmarked" in this case?

Yes, the DRAM dies all come from the same wafer supply and fab capacity, and those limits are the cause of the current prices. However, once the memory OEMs have packaged DRAM dies into something like an LRDIMM or SOCAMM, the cake is baked. It's no longer usable in a laptop or desktop. No amount of X-year-old LRDIMMs (hypothetically) flooding the market will be useful for anyone's desktop or laptop. And then there's HBM, where the dies are directly on-package with the CPU or GPU.

Second-hand, revalidated server DRAM components may contribute somewhat to a price decrease, but those won't be the components you or I will be purchasing (unless you run a true server platform as a desktop, in which case, shine on you crazy diamond!).

The same is partly true for GPUs: there are PCIe versions, but most are OAM or SXM modules. You might be able to jury-rig an SXM module into a desktop? Adapter cards exist for at least some SXM versions, and you could figure out the cooling somehow? But it's probably more trouble than it's worth.

[oompydoompy74]

I have some amount of confidence that the sellers on AliExpress will figure out how to create cheap hardware that makes use out of all of this when it hits the secondary market.

[chao-]

Yes, but also no?

For standard desktop CPUs, the memory controller doesn't support the signaling required to communicate with an RDIMM. There's no clever AliExpress adapter that will magically give a component within your CPU capabilities that it simply doesn't have.

However, if you have a true workstation, you don't even need some adapter from AliExpress! Xeon 600, Threadripper 7000, Threadripper 9000 all support RDIMMs natively.

[wtallis]

If an AI crash left the market with a glut of RDIMMs, we would absolutely see AliExpress sellers removing the individual DRAM packages from used RDIMMs and installing eight out of ten of them onto empty UDIMMs. The overhead costs would be a quick bit of hot air reflow, a 5V PMIC and associated components, and a new module PCB, and the sellers would get to build 10 UDIMMs for every eight RDIMMs they scavenge.

[hnav]

Whenever something like this happens the shortage is exacerbated by hoarders and get rich types. For example used Mac Minis are almost certainly going to find another widespread use or be really underpriced in the secondary market.

[NuclearPM]

Is the a third option?

[thayne]

I wonder if a surplus of these server ram components would lead to consumer hardware designed to make use of them though.

[danparsonson]

Unfortunately most of it will likely be HBM and datacenter GPUs and therefore useless for consumers.

[nixon_why69]

Datacenter GPUs are still self contained and speak over pcie though, right? So its a form factor and cooling problem, not an insurmountable one.

[chao-]

Maybe not insurmountable in the "laws of physics" sense?

However, it may be so costly in cobbled-together parts, and in time to deal with unsupported drivers and/or VBIOS, that it is not worth it compared to using a proper server hardware with proper SXM sockets. Nvidia is also doing a lot of new, exciting things with networking that may make their SXM-based GPUs require even more just-so hardware support as time goes on.

Getting older SXM3 GPUs (i.e. for V100's, from 2017) working via PCIe adapters has been done reliably. However, here is someone who did that successfully, and spent a chunk of time last year trying to do the same for SMX5 (H100) and failing:

https://forums.servethehome.com/index.php?threads/sxm5-h100-...

We were all spoiled by the era from 2005-2020, when you could squint at a "server" configuration and see that it was expensive, high-binned, commodity hardware with some extra RAS and OOBM features. You could buy parts harvested from a retired server based on Xeon E5-1680v3 and drop them in your workstation. Or you could buy an entire single-socket Xeon E5 v3 server, plop in a GPU, and mostly use it as a workstation! As-is!

Now the serious servers have SOCAMM memory, 400Gbps networking, EDSFF SSDs, maybe are configured for CXL 3.0, et cetera. The hardware itself is so divergent that it isn't swap-in-plug-and-play with desktops, even for some high-end workstations.

[nixon_why69]

I really appreciate your comment and the difficulties, but if we're hypothetically talking about $100k hardware for pennies on the dollar.. it looks like the problems are all analog? Like ok, add power supply on top of form factor and cooling but all of the silicon is compatible, right? If (if!) we are talking about pennies on the dollar then those problems are solvable.

For example, if I am a homelab, I don't necessarily need the integrated SFP networking stuff to work, I'm happy with my single overpowered GPU. I don't need CXL either, I just want one badass H200 running in my rig. Maybe Shenzhen will productive that?

[chao-]

There already are SXM5-to-PCIe adapter cards, but from what little I've seen, they are all a bit dodgy. If (if!) there really is some flood of used/decommissioned H100s/H200s into the market, perhaps that will motivate iterations and mass production of these, and they become more reliable?

It is not 100% out of the question. I would also love to see it happen, but am being realistic about the chances, as a check on my own usual optimism.

I think the more likely scenario is, rather than a flood of decommissioned Hopper-era servers being separated into parts, and the parts sold for "pennies on the dollar", we will see entire 4x H100 servers (CPU and RAM included) sold for "dimes on the dollar" to small-and-mid-sized businesses and labs. A formerly-$250K server sold for $50K after six years. Or $20K after ten years. Something like that.

[account42]

They probably won't have the display parts, making them useless as a desktop GPU.

[Zetaphor]

Most are using SXM sockets which would require an adapter to pcie

[broodbucket]

Always the play if you're comfortable sourcing and installing your own because their store will always have a decent markup.

[morserer]

Smart move!

You have lpcamm2 just sitting around in a drawer? Or did you get last-gen?

[InsideOutSanta]

Went with the AMD option for this reason.

[mdhen]

You will not get that 20 hour battery life with the AMD, unfortunately.

[InsideOutSanta]

That's not how I use laptops.