[nathan_douglas]

What's your GPU setup like?

I'm doing a vaguely similar thing - I have a 10" rack minilab [1] and I've vibe-coded an MCP server that runs in the cluster to introspect, etc, but the main longterm goal is to set up some ML pipelines and maybe work toward formal verification via TLA+ or smth. (_not_ vibecoding that... I'm thinking of moving into AI formal verification or compliance automation as a career move.)

I have a separate amd64 server with an RTX 2070 Super - which is obviously old and low-powered. Useful for some general ML stuff, but I don't think it's sufficient to run any non-trivial modern LLM.

I'm thinking about upgrading that GPU, but haven't committed to it or even really thought that hard about it.

[1] https://clog.goldentooth.net/

[vibe42]

The main server runs 3x RTX PRO 6000 (288 GB VRAM combined), power limited to 280W each (can crank it up as temps are fine but about to add some more fans first as the cards are stacked).

The second server is 2x Radeon RX 7900 XTX (48 GB VRAM combined). It's a fairly recent gaming PC that's being repurposed. Idea is to power limit those cards too and run some overnight stuff w small/medium sized models.

Intel just released some 32 GB VRAM cards, but sounds like support across AI tooling is a bit rough atm.

[nathan_douglas]

@#$% that's impressive. A little above my budget. I appreciate your response. I read some of your other comments and will follow your career with interest.