[overrun11]

Gross margins and cost of revenue are well defined accounting terms that apply to any type of business.

> Does it include:

> Inference used for training? Modern training pipelines aren't just gradient descent, there's a ton of inference used in them too.

No because this is training and not inference. Just like how R&D costs for a drug aren't part of COGS either.

> Gradient descent itself?

No

> The CPUs and disks storing and managing the datasets?

Yes

> The web servers?

Yes

> The people paid to swap out failed components at the dc?

Yes to the extent they are swapping for inference and not training. If the same employees do both then the accountants will estimate what percent of their time is dedicated to each and adjust their cost accordingly.

[mike_hearn]

We weren't talking about COGS, we were talking about "cost of compute", which isn't an accounting term.

For the rest, anyone can define and apply an accounting metric but that doesn't mean it tells you anything useful. If you look at the unit cost of any typical IP business it's nearly zero. Yet, many companies lose money on making movies, video games, apps and books.

[torginus]

I'm not familiar with accounting, but I suspect a lot of these cloud infrastructure companies don't throw out hardware for a very long time, just like how AWS sells you their old stuff as whitelabel compute at a markup, behind which I think are mostly old pieces of hardware, I think as long as Anthropic keeps finding uses for the old GPUS provided they dont break, they don't have to write off these assets, which means they don't incur costs using them if they are clever with their books