[miniwark]

2 x Nvidia Tesla P40 card for €660 is not a thing i consider to be "on a budget".

People can play with "small" or "medium" models less powerfull and cheaper cards. A Nvidia Geforce RTX 3060 card with "only" 12Gb VRAM can be found around €200-250 on second hand market (and they are around 300~350 new).

In my opinion, 48Gb of VRAM is overkill to call it "on a budget", for me this setup is nice but it's for semi-professional or professional usage.

There is of course a trade off to use medium or small models, but being "on a budget" is also to do trade off.

[whywhywhywhy]

> A Nvidia Geforce RTX 3060 card with "only" 12Gb VRAM can be found around €200-250 on second hand market

1080Ti might even be a better option, it also has a 12gb model and some reports say it even outperforms the 3060, in non-rtx I presume.

[Eisenstein]

CUDA compute version is a big deal. 1080ti is 6.1. 3060 is 8.6. It also has tensor cores.

Note that CUDA version numbers are confusing, the compute number is a different thing than the runtime/driver version.

[Melatonic]

Not sure what used prices are like these days but the Titan XP (similar to the 1080 ti) is even better

[mock-possum]

Yeesh, yeah, that was my first thought too - who’s budget??

less than $500 total feels more fitting as a ‘budget’ build - €1700 is more along the lines of ‘enthusiast’ or less charitably “I am rich enough to afford expensive hobbies”

If it’s your business and you expect to recoup the cost and write off the cost on your taxes, that’s one thing - but if you’re just looking to run a personal local LLM for funnies, that’s not an accessible price tag.

I suppose “or you could just buy a Mac” should have tipped me off though.