[jonhohle]

Plug it in, get the cuda drivers, run ollama or equivalent, and go from there. I use an 4GB Quadro card from the same generation and depending on the model can get faster than I can read output. It’s not as good as commercial models, but smaller Qwens and Gemmas work well. The Quadro came with a Dell refurb that cost about as much as the card would have by itself at the time.

(I’m doing this with passthrough on a FreeBSD host with bhyve to a Linux guest. Works great.)

I’d like a beefier card, but until prices come down, even older cards like this can do some interesting things if you give them enough time.