[CamperBob2]

I'd just get an RTX Pro 6000 Blackwell and call it a day. More VRAM. Somewhat less bandwidth but it's your bandwidth.

[k2so]

Yeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.