[hvb2]

The tech is way underpriced right now. It's basically a subsidized market right now, with the money flowing in coming from the private sector.

The problem here is that it remains to be seen who is willing to pay for the service once it's priced at cost or even with a margin. And based on valuations of AI companies one would expect a huge margin.

[ForHackernews]

It's hard for me to imagine paying real money for something that gives me a maybe-hallucinated answer that I need to check every single time. A flaky test is worse than a failing test.

[rwmj]

Plus I can run a reasonable LLM on my own hardware, so I don't even need to pay anyone else. And what I can run locally is only going to get better and better.

[keeda]

This is true, but this is also true for on-premise hosting vs cloud. And cloud has been booming for at least a decade before LLMs appeared. I suspect AI will follow a similar trajectory, i.e. companies don't move their AI deployments on-prem until they hit a certain scale.

[rwmj]

This is very true, but I think the other point is that AI doesn't have much "moat". If a competitor can take a pre-trained Chinese LLM, fine tune it a bit, fiddle with the prompt, and ship a product which is not as good but way cheaper, then you've (or Oracle's) got a problem.

[keeda]

Actually, in that scenario the AI labs (OpenAI, Anthropic, etc) have a problem. The cloud providers (including Oracle!) will do with the models what they've been doing with open source software: just take it and run it on their infra and charge money for providing it as-a-service.

This is why you're seeing the AI labs now try to build their own data centers.

[danielbln]

_sigh_

Yes LLMs hallucinate, no it's no longer 2022 and ChatGPT (gpt-3.5) is the pinnacle of LLM tech. Modern LLMs in an agentic loop can self correct, you still need to be on guard but if used correctly (yes, yes, holding it wrong etc. etc.) can do many, many tasks that do not suffer from "need to check every single time".

[chmod775]

I must be holding it wrong then, because in my ChatGPT history I've abandoned 2/3rds of my conversations recently because it wasn't coming up with anything useful.

Granted, most of that was debugging some rather complicated typescript types in a custom JSX namespace, which would probably be considered hard even for most humans as well as there being comparatively few resources on it to be found online, but the issue is that overall it wasted more of my time than it saved with its confidently wrong answers.

When I look at my history I don't see anything that would be worth twenty bucks - what I see makes me think that I should be the one getting paid.

[topaz0]

I think the reason people talk past each other on this is that some of them are using LLMs for every little question they have, and others are using them only for questions that they can't trivially answer some other way. Sure, if all your questions have straightforward, uncontroversial answers then the LLMs will often find them on the first try, but on the other hand you'd also find them on the first try on wikipedia, or the man page, or a google search. You'll only think the ChatGPT is useful if you've forgotten how to use the web.

If you're only asking genuinely difficult questions, then you need to check every single time. And it's worse, because for genuinely difficult questions, it's often just as hard to check whether it's giving garbage as it would have been to learn enough to answer the question in the first place.

[coffeebeqn]

If a coworker is wrong 40% or 60% of the time I’ll ignore their suggestion either way

[danielbln]

As you should, but an LLM is not a human, nor is it categorically 40-60% wrong, so I'm not sure what your point is.

[georgemcbay]

> Modern LLMs in an agentic loop can self correct

If the problem as stated is "Performing an LLM query at newly inflated cost $X is an iffy value proposition because I'm not sure if it will give me a correct answer" then I don't see how "use a tool that keeps generating queries until it gets it right" (which seems like it is basically what you are advocating for) is the solution.

I mean, yeah, the result will be more correct answers than if you just made one-off queries to the LLM, but the costs spiral out of control even faster because the agent is going to be generating more costly queries to reach that answer.

[refulgentis]

Apologies that you're taking on the chin here. Generally, I'll just skip fantastical HN threads with a critical mass of BS like this, with pity, rather than an attempt to share (for more on that c.f. https://news.ycombinator.com/item?id=45929335)

Been on HN 16 years and never seen anything like the pack of people who will come out to tell you it doesn't work and they'll never pay for it and it's wrong 50% of the time, etc.

Was at dinner with an MD a few nights back and we were riffing on this, came to the conclusion is was really fun for CS people when the idea was AI would replace radiologists, but when the first to be mowed down are the keyboard monkeys, well, it's personal and you get people who are years into a cognitive dissonance thing now.

[Libidinalecon]

I just totally disagree.

I want AI to be as strong as possible. I want AGI, I especially want super intelligence. I will figure out a new and better job if you give me super intelligence.

The problem is not cognitive dissonance, the problem is we don't have what we are pretending we have.

We have the dot com bubble but with a bunch of Gopher servers and the web browser as this theoretical idea yet to be invented and that is the bull case. The bear case is we have the dot com bubble but still haven't figured out how to build the actual internet. Massive investment in rotary phone capacity because everyone in the future is going to be using so much phone dial up bandwidth when we finally figure out how to build the internet.

[danielbln]

Yeah, it really pulled the veil away, didn't it? So much dismissiveness and uninformed takes, from a crowd that had been driving automation forward for years and years and you'd think they'd get more familiar with these new class of tools, warts and all.

[Libidinalecon]

I just can't understand how anyone who actually uses the tools all the time can say this.

[danielbln]

Say what exactly? Driving automation of all kind with Claude Code level tools has been incredibly fruitful. And once you spent sufficient time with them you know when and where they fall on their faces and when they provide real tangible reproducible benefits. I could not care less for the AI hype or bubble or whatever, I just use what I see works as I'm staring these tools down for 10h+/day.

The problem is that these conversations are increasingly drifting apart as everyone has different priors and experiences with this stuff. Some are stuck in 2023, some have so very specialized tasks that it's more work whipping the agent in line that it saves, others found a ton of automation cases where this stuff provides clear net benefits.

Don't care for AGI, AI girlfriends or LLM slop, but strap 'em in a loop and build a cage for them to operate in without lobotomizing themselves and there's absolutely something to be gained there (for me, at least).

[dwa3592]

really? >>many tasks that do not suffer from "need to check every single time"

like which tasks?

How do you decide whether you need to check or not?

If you're asking it to complete 100 sequences, and if the error rate is 5%, which 5% of the sequences do you think it messed up or _thought_ otherwise? if the 5% is in the middle, would the next 50 sequences be okay?

[palmotea]

> really? >>many tasks that do not suffer from "need to check every single time"

> like which tasks?

Making slop.

[hamburglar]

If I ask an LLM to guess what number I’m thinking of and it’s wrong 99.9% of the time, the error is not in the LLM.

[causal]

I wonder if a price correction would be a boon for open source, with the economics of smaller / self hosted models making a lot more sense when API prices have to surge.

[refulgentis]

It's not actually subsidized and the economics of smaller/self-hosted models are a much, much, worse nightmare (source: guy who spent last 2 years maintaining llama.cpp && any provider you can think of) (why is it bad? same reason why 20 cars vs. 1 bus is bad. same reason why only being able to use transportation if you own a car would be bad)

[causal]

> It's not actually subsidized

Source?

[anoncareer0212]

Source on it being subsidized? :) (there isn't one, other than an aggro subset of people lying to eachother that somehow literally everyone is losing money, while posting record profit margins) (https://en.wikipedia.org/wiki/Hitchens%27s_razor)

[causal]

If it's not profitable, it's running on capital. Subsidized.

[conartist6]

And really the reason that it would be like that is that the models don't learn, per se, within their lifetime.

I'm told that each model is cashflow positive over its lifetime, which suggests that if the companies could just stop training new models the money would come raining down.

If they have to keep training new models though to keep pace with the changes in the world though then token costs would be only maybe 30% electricity and 70% model depreciation -- i.e. the costs of training the next generation of model so that model users don't become stranded 10 years in the past.

[mistrial9]

> it remains to be seen who is forced to pay

via govt relationships, long term irreplaceable services, debt or convictions.. Also don't forget the surveillance budgets and the best spigots there, win.

[refulgentis]

It's not subsidized, lol.

Generally, I worry HN is in a dark place with this stuff - look how this thread goes, ex. descendant of yours is at "Why would I ever pay for this when it hallucinates." I don't understand how you can be a software engineer and afford to have opinions like that. I'm worried for those who do, genuinely, I hope transitions out there are slow enough, due to obstinance, that they're not cast out suddenly without the skills to get something else.

[philipwhiuk]

> It's not subsidized, lol.

It's subsidised by VC funding. At some point the gravy train stops and they have to pivot to profit so that the VCs deliver return-on-investment. Look at Facebook shoving in adverts, Uber jacking up the price, etc.

> I don't understand how you can be a software engineer and afford to have opinions like that

I don't know how you can afford not to realise that there's a fixed value prop here for the current behaviour and that it's potentially not as high as it needs to be for OpenAI to turn a profit.

OpenAI's ridiculous investment ability is based on a future potential it probably will never hit. Assuming it does not, the whole stack of cards falls down real quick.

(You can Ctrl-C/Ctrl-V OpenAI for all the big AI providers)

[anoncareer0212]

This is all about OpenAI, not about AI being subsidized...with some sort of directive to copy/paste "OpenAI" for all the big AI providers? (presumably you meant s/OpenAI/$PROVIDER?)

If that's what you meant: Google. Boom.

Also, perhaps you're a bit new to industry, but that's how these things go. They burn a lot of capital building it out b/c they can always fire everyone and just serve at cost -- i.e. subsidizing business development is different from subsiziding inference, unless you're just sort of confused and angry at the whole situation and it all collapses into everyone's losing money and no one will admit it.

[hvb2]

You're replying to a story about a hyperscaler worrying investors about how much they're leveraging themselves for a small number of companies.

From the article: > OpenAI faces questions about how it plans to meet its commitments to spend $1.4tn on AI infrastructure over the next eight years.

Someone needs to pay for that 1.4 trillion, that's 2/3 of what Microsoft makes this year. If you think they'll make that from revenue, that's fine. I don't. And that's just the infra.

[mike_hearn]

I'm a big fan and user of AI but I don't see how you can say it's not subsidized. You can't just ignore the costs of training or staff or marketing or non-model software dev. The price charged for inference has to ultimately cover all those things + margin.

Also, the leaked numbers being sent to Ed Zitron suggest that even inferencing is underwater on a cost basis, at least for OpenAI. I know Anthropic claims otherwise for themselves.

[AnimalMuppet]

A huge margin or a huge market at a moderate margin. But yes, the net profit has to be huge.

[energy123]

You're saying the unit economics are bad?