Hacker News new | ask | show | jobs
by altcognito 13 days ago
> compute/manufacturing capacity for current nodes which doesn’t age nearly as well

I mean, compute depreciates, but I think there is zero chance that the value of inference or training is going to fall to zero. Market discovery will find the right price provided the market has the right degree of freedom. Given the type of market it is, I don't see how that won't be the case.
3 comments
jaggederest 13 days ago
I'm a big fan stylistically of what https://taalas.com/ is doing, as far as models baked into silicon. If you haven't tried their chat it's absurdly fast (and also very very dumb)

That implies to me that in the future we'll have models as good or perhaps better than the state of the art at the moment, but on hardware chips that can be put in places where you can't currently locate a datacenter, and operating at hundreds of times better power efficiency, which sounds pretty great.

link
lambdaone 13 days ago
Algorthmic improvements in inference could make all that kit redundant very quickly - there are already moderately capable models that can be run on phones or laptops with specifications that are currently high-end but will be mainstream in another year or so.

This will lead to a superabundance of power-hungry compute power in the hyperscalers, and it's not entirely clear what can be done to consume it all and still run at a profit unless they manage to make ever greater gains for ever more compute-hungry models that cannot be run on consumer devices, unless they refresh their hardware at ever faster and more expensive rates.

The joke about data centers used to be that their core business was selling power at a loss; this may end up being true of the hyperscalers next.

link
Retric 13 days ago
That hardware costs GW of electricity at scale. So barring major disruption in R&D you hit a cost curve cliff where new hardware is simply more cost effective even if existing hardware is free.

Some workloads may make sense running for a few hours a day during cheap solar prices on outdated hardware, but in less than a decade the value is very much hitting zero.

link