[brianjking]

This particular model has 83.66gb of model weights so you'll need to 2x Nvidia 80gb A100 at a minimum unless you're loading it in 8bit mode.

[brianjking]

With that said, there are ggml/gptq and other optimization techniques.