[bayesianbot]

That is exciting!

I don't understand how DeepSeek can be so cheap with their cache pricing - ~0.003 usd / 1Mtok. 100x less than Kimi K3, or similar numbers against pretty much any other decently sized model to my knowledge. I've been using it whenever possible as even longer agent sessions cost few cents.

[sudosysgen]

If you read DeepSeek's papers, you'll find a litany of architectural features that allow for a greatly reduced cache hit price by shrinking the size of the KV-cache.

[yfontana]

How come no other big model seems to be able to deliver the same type of extremely low cache cost though, if their techniques are public?

[SXX]

Why dont you think proprietary models use these optimization techniques and way more?

It very much possible Anthropic, OpenAI and Google able to serve their models much cheaper than their current API prices.

They just dont do it because they dont try to ubdercut each other and so far chinese models been percieved behind SOTA.

[jboss10]

I think the "architectural features" are part of the model, not the kv cache. So implementing it would be difficult and expensive.

[natrys]

They can and almost certainly are doing similarly impressive engineering works internally.

They just aren't in any hurry to forward those cost savings to you.

[sudosysgen]

Many of these techniques haven't been published very long ago - it often takes a good 6-8 months for techniques to percolate. But also, they come at a complexity cost and, seemingly, also at a stability cost.

[hnfong]

Also potentially a performance (in terms of output quality) cost. DeepSeek is cheap on a per token basis but lags behind in the benchmarks, perhaps it was a calculated tradeoff.

[petu]

To be fair Deepseek V4 now is just a preview, which likely implies training isn't completed.

[petu]

Deepseek V4 paper is just ~three months old

[hack1312]

What provider are you using?

[bayesianbot]

DeepSeek's own API

[greyb]

Any way to avoid China sales tax or is that just the cost of doing business?

[NortySpock]

https://openrouter.ai/deepseek/deepseek-v4-pro#providers

Look through the provider list for a company you are willing to do business with?

[anigbrowl]

Good grief, the sales tax is only 6% on a service that's already extremely affordable.

[verdverm]

Fireworks.ai

[greyb]

I think I'd rather pay Chinese sales tax than work with Fireworks, but thank you for the suggestion.

[sourcecodeplz]

it is ridiculous really. it is so cheap, that i can just run it basically 24/7.