[elcritch]

Yesterday I did some testing on the cost to solve the same simple problem on openrouter with different models using cline. Simple problem but it had a few nuances to solve it properly and so required reasoning.

After reading comments like this I was expecting (hoping?) that DeepSeek or similar would be cheaper.

However I was surprised that DeepSeek v4 cost about 5.5x GPT-5.4 to solve the problem.

- Deepseek-v4-pro-medium cost $2.47 - GPT-5.4-medium cost $0.45 - GPT-5.5-low was $0.86

[VulgarExigency]

That doesn't sound right. Were you using the actual Deepseek provider? The one time I spent 3 dollars on Deepseek in a day, I had 615k output tokens, 96M cache hit input tokens, and 5M cache miss output tokens.

[nl]

It's not unheard of for "more expensive" models (on a per-token basis) to end up cheaper than weaker models (on a per-task basis).

Kimi K2.5 is roughly double the price (per token) of DeepSeek v4 Pro, but cost $0.05 vs $0.16 (for the same score) on my own benchmark.

https://sql-benchmark.nicklothian.com/?highlight=moonshotai_...

https://sql-benchmark.nicklothian.com/?highlight=deepseek_de...

[HDBaseT]

Yeah, I struggle to use more than a few dollars a day using Deepseek V4 Pro (max reasoning).

* Some people suggest not using max reasoning due to overthinking and looping issues, this may consume more tokens than needed.