[rakel_rakel]

The hyperbolic nature of the articles in both AI camps is very exhausting to me.

I'd like to get in front of a whiteboard with someone who knows economics and the token providers businesses well enough to answer my "explain to me like I'm five" questions. But I'll start with these in here:

Is my observation correct that for the token providers this is a margins game, while for the consumers this is a quality of service/product game? If the quality:margin lines will cross at some point on the x-axis, is the race is to reach this point before running out of money? If yes: What historical examples are there where the delta between these two is huge?

I'm guessing LLM's are unique in a sense, since there's really no limit to how good a consumer of the product expects it to get? (Compared to for example email which is much easier to scale in regards to compute.)

Also extreme noob at life question: Why would you want to IPO before having a sustainable business model? What's the upside?

[rstuart4133]

You can ask an AI what it costs to produce an Opus token. The answer I got was $0.216/M Token. That takes into account hardware cost, power, cooling and hosting.

Others have posted links here saying inference is about 30% of Anthropic's spend, the other 70% is R&D - things like developing the next model. If you take that into account, add a profit margin, and round to make the figures easy you end up with them selling tokens at $1 / M Token to make money.

Their API cost is currently $25 / M Token. There is no question that's profitable. Someone really pushing their $200/mo Max plan can use 10B Tokens per month, which works out at $0.02 / M Token, so they are eating a huge loss there. That is clearly going to go away at some stage. For the rest of us: look at your average monthly token usage. If you are using 100M Tokens a month with the Anthropic $100/mo plan, they are making money out of you.

You have to be working very long hours, and be really, really proficient at using AI to achieve 10B Tokens a month. The only way to be that proficient is to have been using it for a long time, years in fact, so it was useful to you long before Opus came along. You would be very disappointed to lose Opus, of course - but you are just the sort of person who can make a less capable model sing. It's not so difficult to see those users moving to an in-house-hosted, open-source model in a few years, and it will cost them what they are paying now - $0.02 / M Token.

All that means I'm not convinced by the gloom and doom vibe of the article. Things will change, but it won't mean the end of AI usage.

[rakel_rakel]

That's helpful, thanks!

Once past IPO, I suspect the 70% -> R&D must shrink, right? I mean, to keep the stock afloat long term P/E must come down right?

Public investors strike me as less willing to pour money into R&D, which is why I'm wondering about the timing of IPO in my initial question.

[rstuart4133]

> Once past IPO, I suspect the 70% -> R&D must shrink, right? I mean, to keep the stock afloat long term P/E must come down right?

I dunno. Yes, the stock market dictates they get a return. But the return they get on that R&D isn't easy to determine. The return is high now. When you go from the previous best model only being able to find a handful of security issues, to the next model producing a flood so large Linus is screaming rage into the void complaining that his legion of helpers can't handle the flood of new CVEs (but he can't really blame anyone for this), development is going very fast. They haven't monetised the ability yet, but really does critical infrastructure have any choice now other than to get their entire codebase vetted by Mythos? How much is that monopoly worth? (Granted - it won't last long.)

Their R&D is aimed at discovering - well I'm not sure what you would call it, but it's the same thing that allows a mother to take a glance at her child and know what's troubling them, or a senior engineer to glance at code and spot a bug, or a same thing that a chess master to look at a chess board to produce a move in a second or so that defeats all but the best chess players. There are lots of these areas of expertise to be discovered. They are worth a lot of money. When does it stop?

A hyped exploration of this by bloomberge: https://www.youtube.com/watch?v=JmFKaqJg5X4