[lardosaurusrex]

I think this fails to take into account how many people are fine with "fast enough" vs "fastest".

I've seen people happily use AI that takes several minutes to generate text or edit an image because to them they already aren't using their computer when they tell it to start; they just grab their phone and walk away and come back only to check in on it.

I feel like people here and on other technology discussions -- although it's worse here -- don't seem to parse what being the minority means.

They know they're one of the few to have access to such incredible hardware -- whether it be rented or purchased for way too much cash -- but they only see their own kin; their own ilk. They only compare themselves to the best.

The reality is that nobody expects data centre speed nor power in their own home and are satisfied to just go "haha its thinking" and let their computer quietly tick in the background as opposed to paying outragious prices for subscriptions or hardware.

[gmadsen]

That is completely discounting future capabilities and new use cases. Sure in 10 years you will have current SOTA locally, but in no way is it obvious we are anywhere near the limit of marginal value from improved capability

[lardosaurusrex]

Unless you're a developer or doing complicated research you do not need much to use an LLM at home with consumer-grade hardware.

I'm editing photos with 32GB of DDR4 RAM at 3200mhz alongside a 3050 with a nearly decade-old mobo and ryzen 5800XT and at most? It takes 30 seconds and that's allowing the gpu to use my actual RAM as 'fallback' memory.

There are photoshop filters -now- that take longer than that before the LLM craze even began. Hell; I can train a lora in an hour or two.

This obsession with "more specs more data faster and faster" isn't going to win; it already isn't winning.

Deal with it or get wrecked.