[boesboes]

What costs are 1/100th?

[fooker]

Of serving a (approximately) gpt4 sized model.

[usrusr]

What made the cost go down? Can't be cheaper used H100, can't be cheaper RAM. A revolutionary breakthrough in hardware use per query?

[barumrho]

Is this true? Hardware costs have only gone up during this time. Are you referring to electricity cost to serve these models? (i.e. compute got more efficient?)

[underlipton]

Does that include the capital costs of spinning up to the current models/scale or is it just running costs?

Also, lost revenue from other services being degraded by shifting resources to supporting training/serving models (Google Search...)?

[alangibson]

So the number is irrelevant. No one wants yesterdays newspaper.

The only relevant number is the price to serve a frontier or near-frontier model.