[ismailmaj]

it's 80% less cost, not 80% in efficiency gains, could be that Luna was overpriced to begin with, we don't have much info on the models themselves.

Assuming the efficiency gains are real, I feel like something has to give, maybe worse quality due to aggressive quantization/kv cache compression?

[heisgone]

Let's suppose each models was subsidized at 70%, so that we only pay 30% of the cost. They would loose much more money per token on the more powerful models. It's in their interest to encourage the use of the less expensive models. Let's say they increase Luna subsidies at 90%. They would still "save" relative to the use of the more expensive models.

[anthonypasq]

> Let's suppose each models was subsidized at 70%, so that we only pay 30% of the cost.

why on earth would you suppose that?

[oblio]

Because their leaked financials point that way.

[gruez]

Source?

[oblio]

https://www.wheresyoured.at/exclusive-openai-financials/

[beering]

The company losing money does not mean the model inference in API is 70% subsidized- that’s a crazy leap in logic. Obviously the massive number of free chatgpt users are getting subsidized 100%.

[mgambati]

Not true at all. Do you really believe that generating 1m output tokens on sol really cost 100 usd?

These companies are printing money on inference. The only issue is the capex of expending on new servers but they are generating TONS of income.

[claw-el]

If the frontier labs are indeed facing threats from open weight models, we would first see this in pricing pressure from the base model (like Luna) first.

[mlinsey]

High-performing open weight models being released recently, and your customers looking into working with multiple providers as a result, are a great reason to drop prices on your non-frontier offerings.

Although I'm sure there are some efficiency gains, the technology is too new and labs are scrambling to release too quickly to think that the low-hanging optimization fruit has been picked already.

[dannyw]

Been using OpenAI models since ada/babbage/curie/davinci and at least from my own experience, their APIs feel the same.

If you use Codex it's different, the harness has a lot to do with it and there's definitely been changes including recently.

[axus]

Something can be overpriced and still lose money.