[Jedd]

In January 2024 there was a similar post ( https://news.ycombinator.com/item?id=38985152 ) wherein the author selected dual NVidia 4060 Ti's for an at-home-LLM-with-voice-control -- because they were the cheapest cost per GB of well-supported VRAM at the time.

(They probably still are, or at least pretty close to it.)

That informed my decision shortly after, when I built something similar - that video card model was widely panned by gamers (or more accurately, gamer 'influencers'), but it was an excellent choice if you wanted 16GB of VRAM with relatively low power draw (150W peak).

TFA doesn't say where they are, or what currency they're using (which implies the hubris of a North American) - at which point that pricing for a second hand, smaller-capacity, higher-power-drawing 4070 just seems weird.

Appreciate the 'on a budget' aspect, it just seems like an objectively worse path, as upgrades are going to require replacement, rather than augment.

As per other comments here, 32 / 12 is going to be really limiting. Yes - lower parameter / smaller-quant models are becoming more capable, but at the same time we're seeing increasing interest in larger context for these at home use cases, and that chews up memory real fast.

[T-A]

> TFA doesn't say where they are

"the 1,440W limit on wall outlets in California" is a pretty good hint.

[dcassett]

San Francisco specifically:

"I prompted ChatGPT to give me recommendations. Prompt: ... The final build will be located at my residence in San Francisco, CA, ..."

[zxexz]

Bringing back memories of testing the breakers in my college apartments to verify exactly which outlets were on which circuit, so I could pool as much as possible as needed. I distinctly remember pulling 20kw once, celebrating with a beer; the memory of all those cables snaking through the old apartment makes me almost uneasy now. I do remember we didn’t have to pay for heat that winter; which felt like a major win in Massachusetts. Come to think of it, I’m pretty sure there are still some servers tucked away in a crawlspace in that basement.

[throwaway314155]

> which implies the hubris of a North American

No need for that.

[Jedd]

Probably true.

But for those of us outside the USA bubble, it's incredibly tring to have to intuit geo information (when geo information would add to the understanding).

As others noted in sibling comments, TFA had in fact mentioned in passing their location (in their quoted prompt to chatgpt, and at the very end of the third supporting point for the decision to go for an Nvidia 4070) 'California, CA'. I confess that I skimmed over both those paragraphs.

Now, sure, CA is a country code, but I stand corrected that the author completely hid their location. Had I spotted those clues I'd not have to have made any assumptions around wall power capabilities & costs, new & second hand market availability / costs, etc.

I think I mostly catered for those considerations in the rest of my original comment though - asserted power sensitivity makes it surprising that a higher-power-requiring, smaller-RAM-capacity, more-expensive-than-a-sibling-generation-16GB card was selected.

[topato]

True, though

[topato]

He did soften the blow by saying North American, rather than the more correctly appropos, American

[dfc]

The author also refers to Californian power limits. So it seems the criticism is misplaced.

[1shooner]

>TFA doesn't say where they are, or what currency they're using

They say California, and I'm seeing the dollar amount in the title and metadata as $1,3k, was that an edit?