[JimDabell]

Yes. It’s already profitable to run inference at today’s prices. AWS isn’t subsidising you when you buy compute from them. And inference cost is declining steeply.

> The cost of LLM inference has dropped by a factor of 1,000 in 3 years.

— https://a16z.com/llmflation-llm-inference-cost/

AI startups are not profitable because they are throwing vast sums of money at growth and R&D, not because inference is unaffordable.

[anonzzzies]

But, we need a future where unlimited inference, in parallel is profitable. It is not: even less than cloud compute (where it is terrible also), when I buy 500 flimflams for $50/mo, what did I buy exactly? As currently it seems to depend on the position of the moon: one time 10 prompts make what I want, sometimes 100 prompts keep looping over the same issue unable to fix it (like a typescript type issue which takes me 1 seconds, llms, the flagship ones, can easily burn 100 prompts and not fix it). I do very much NOT want to pay for those 100. I see 'vibecoders' aka people who cannot code, burn through all Tokens for the month without having anything working in a single day.

[JimDabell]

The question that was raised was whether or not current LLM usage will be affordable after providers decide to be profitable.

You are asking if infinite usage is affordable.

[jama211]

A bit of an out of context reply for me to jump in here, but in the abstract, it can be a reasonable question to ask if infinite usage is affordable. Maybe not infinite without constraints… but as an example from the past there are many mobile phone plans that have “infinite” calls and texts for an affordable monthly cost. There would’ve been a time where asking if unlimited calls would be affordable would’ve sounded insane, but now it’s fairly normal.