[phaser]

What intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop.

Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll be able to run locally in 6 years will be very exciting. or frightening.

Also I don’t know what this means for the valuation of the AI companies. I remember asking about this very idea to one of their employees at an event and instead of answering he bailed out to grab a cocktail.

[MAXPOOL]

Things you are not supposed to talk about:

- There is no "moat" (lasting, easy-to-defend technological edge) in AI model businesses. There are just short-term advantages.

- An AI business is a capital-intensive business, just like old factories. Data centers are expensive, models are energy-hungry, and the hardware inside must be replaced every 3–4 years.

- Smaller, specialized models eat margins from below. Transcription, voice, or image detection do not need large models.

There is no reason to expect high margins like you can in traditional software business. Benefits of AI go mostly to consumers.

edit: There is potential for economies of scale. Few megacorps can strive for cost advantage when they achieve scale (Microsoft, Google, Amazon and Meta)

[twoodfin]

All true.

It does seem like the structural characteristics we’ve observed so far suggest there is a kind of flywheel from short-term to long-term advantage due to the capital requirements at various levels.

If you’re Nvidia, making the best GPUs today, the expanding wavefront of demand is consuming them with volume and margins to give you a huge edge in building out the best next generation of GPUs. Similar to how the mobile wave gave TSMC sustained advantage for about a decade now.

I’m guessing this is also what we’re seeing as Anthropic and OpenAI swap spots in the token-vendor market.

[DrScientist]

I can see the fly wheel in action for Nvidia[1], but in terms of model building - I think the companies that have the advantage here are not Anthropic or OpenAI, but rather companies with substantial revenues from other sources - Google is the obvious player here - reported to be planning on spending 185 billion this year without having a raise a dime from the markets, but there are plenty of other companies - like Meta or Alibaba who can easily fund the longer game from existing revenues.

[treis]

Everybody talks about this stuff all the time

[fooker]

What you can run locally in consumer hardware is progressing pretty well.

If you get a not-quite-the-best gaming GPU like a 5080, you can run local models that are better than the state of the art from early 2025. Depending on what you want to do, you might have to switch models. The one size fits all huge models are still a data center thing.

[skdb476]

Its a convenience thing. You can run a whole lot of stuff locally from wikipedia to social media/email/video servers whatever. Most people with a full time job and 2 kids dont do it cause who has time and energy to patch and maintain the ever growing complexity of this stuff. These systems will keep growing complex. That also means more bugs. Age old tradeoff between freedom and convenience.

[phaser]

You can run mediawiki at home but you won't have wikipedia. You can run a video server but you won't have all the movies that Netfix has. A local model is actually the real thing.

[skdb476]

you can have the whole wiki loaded with full search available locally. check out kiwix.

[phaser]

Thanks I didn't know about kiwix, but, let's consider the fact that a wiki, or netflix movies are cheap or free, while AI is actually quite expensive at least for now, and i'm not sure if it's because of real costs or to justify the valuation.

So there is a bigger incentive to run locally something that's gonna get you $20 or $100 worth of bills to OpenAI than to mirror something that is actually free.

Example: In the past there was a whole market for sound cards, if you wanted your computer to have any "multimedia" capabilities you needed to get a sound blaster but now everybody assumes a computer will produce sound, and it's basically for free as all chips have it. Now sound interfaces are still a thing but only for audiophiles who are esoteric enough like me to believe that it's worth to have that extra hi-fi quality.

What I think it could happen, is that eventually AI will be part of all the chips, just like soundcards. And there will be people who will buy specialized AI from companies that perhaps are not OpenAI or Anthropic but second-generation sleepers who watched the carnage in the market and decided to enter when it was reasonable.

This could be Apple, or Nvidia or something new. They're just waiting for the others to do the research and introduce the taste for it to the masses, just like sound blaster made us fall in love with high fidelity sound in our computers.

[clusterhacks]

--what this means for the valuation of the AI companies

Probably nothing. Most users have no idea what an LLM is or how it runs. Anecdotally speaking, I see many LLM users default to whatever their day job provides to them. And even slightly more sophisticated users seem ok with paying for their openai or anthropic subscriptions.

Maybe we will see a small but dedicated group of open weight model users who prefer local llm, but everybody else will just consume from the big providers? The scenario might look something like OS choices today - a small, committed group of Linux users vs the vast majority of other users running Windows, MacOS, or Chrome?

[exhilaration]

Prices from OpenAI and Anthropic have really jumped in the past month. I work for a big giant company and our Github co-pilot costs increased as of today, June 1st. Our internal estimates are that our bill will double or triple. How much are we willing to pay? I don't know, but nobody wants to be "left behind".

I think there's actually a big market opportunity here. Somebody, like Dell or HP, should start selling turnkey on-prem LLM servers.

[mr_toad]

This has always been true of software, particularly games. You can get a 5-6 year old game for a fraction of the price, and run it on modest hardware. But the industry wont sit on its hands for 5 years, there will be newer software that requires better hardware.

[phaser]

Technology doesn't always work like that.

A new game is a totally new world with everything created from scratch. A creation. A model, on the other hand, is a reinterpretation machine for hundreds of years of human creations, but not a creation in itself, more like a discovery.

You would think that by now we would have a much better Bitcoin that's taking over the payment networks of the world but what we actually got is a shitload of shitcoin.

[rienbdj]

Training AI models to drive valuation reminds me of high frequency trading