[samnco]

Yes, each GPUs has a 4x -> 16x and a 4x-4x extender, in addition to the m.2 -> PCI-e 4x adapter.

So many potential failure points in there. The sole use case is CUDA. Essentially I wanted a portable cluster with GPUs and that did the work for a couple of month. Now it's getting more serious so the switch to T630 makes sense, and I repurposed the NUCs into the control plane of the K8s cluster.

[jacquesm]

I built this long ago:

https://clustercompute.com/images/image4.jpg

Which was a lot of fun.

Do you have all the GPUs internal to the T630?

Any chance of a picture (of the guts)?

I'm seriously thinking of duplicating your effort.

[samnco]

Here you go :) https://drive.google.com/file/d/0B1CCk51NQ4koSmkxSmxWb1E5Y0E...

Replicating is not very hard. You need a lightweight x86 machine for MAAS, which takes ~20min to install, one VLAN for the iDRAC (IPMI), another for networking that can connect to internet, and off you go. You can also enable KVM power management in MAAS to run the Juju control plane in VMs and save a box if you're limited in compute power.

https://maas.io https://jujucharms.com/docs https://www.ubuntu.com/containers/kubernetes for all the goodies.

If you run into problems, I am SaMnCo on #juju in freenode.

[jacquesm]

Ok, so 2 GPUs in there. Have you tried 4 or is that not possible for some reason?

I have plenty of other hardware floating around here so no problem on hooking it all up.

Thank you for the image.

[samnco]

I have not, but it is technically possible. the PSU is the double 1100W with the GPU enablement kit. Up to 4x PCI-x 16x full speed. Also up to 1.5TB RAM, and 8x 3.5" HDD or 16x 2.5". I didn't go this far though ($$$...)