[rainclouds]

Hardware requirements? I think I can hit the memory bandwidth building from parts I have in my house. Maybe even 2x. Asking for fun not profit.

[morphle]

I'd love to visit your house then. You have 768-1400 GB DRAM with 6000 GB/s memory bandwidth? Nice house.

In my house I currently have almost 900 GB/S memory bandwidth in aggregate but only 132 GB total DRAM.

[rainclouds]

I’ve got a terabyte of DDR4, and a bunch of old thread rippers. They can take 256 each and 8 channel.

[morphle]

Yep, that's the right stuff. Now simply cross-connect all the free PCIe lanes of all thread rippers and you have a nice cluster for LLMs.