[teleforce]

>DECISION: Nvidia RTX 4070

I'm curiuos why OP didn't go for the more recent Nvidia RTX 4060 Ti with 16 GB VRAM that cost cheaper (~USD500) brand new and lesser power consumption at 165W [1].

[1] RTX 5060 Ti 16GB sucks for gaming, but seems like a diamond in the rough for AI:

https://news.ycombinator.com/item?id=44196991

[qingcharles]

And if you're gonna be fine with 12GB, why not a 2080ti instead?

[Fluorescence]

Only 11GB... but I guess it will allow you to not do anything useful just as well as 12GB will :)

You can however solder on double-capacity memory chips to get 22GB:

https://forums.overclockers.com.au/threads/double-your-gpu-m...

I hoped the article would be more along these lines than calling an unremarkable second-hand last-gen gaming pc an "AI Server".

[iamnotagenius]

4060ti has abysmal bandwidth 288 Gb/sec which is a no go for llms.