[lmf4lol]

We rolled out Deepseek V4 Flash to our customers and it was an absolute disaster, unfortunately. It was not able to follow simple commands, always "forgot" to do things, lied consistently about its work, and so on. It was pretty good though on on-off work, like summarizing something or executing simple commands, so we are experimenting now with using it for subagent work with clear instructions and hand off.

Deepseek V4 Pro on the other hand is a really really good main driver and we have a lot of success using it. Its not Opus or GPT-5.5 level but on its way. Kimi 2.6 as well btw.. so there is already quite some choice.

[mark_l_watson]

Your experience with DeepSeek v4 Flash differs from mine: while I usually use DeepSeek v4 Pro (that is also inexpensive), I find using DeepSeek v4 Flash with the Fireworks.ai API and properly configured OpenCode to be very good for routine work, and it is pleasantly very fast. Admittedly I use DeepSeek v4 Pro for difficult problems.

I encourage people to at least once a month to do a quick evaluation with their own problems and workflows. Estimate cost as both what inference tokens cost for a task and also how much human effort it takes to get required results.

I disregard benchmarks.

[lmf4lol]

We are also using fireworks as our model provider. Our harness is openClaw, so tasks are not only coding but all kinds of tasks. For instance, I asked to fetch some info from the web via Chrome browser and to collect the info in an MD. The MD never appeared, even though it claimed to. I asked it three times to write the MD and it was always: “oh yes, I do it now..” then nothing. The search itself also was very bad because it just gave up after one page and hallucinated an answer and - even worse :-) - told me it was very thorough…

Pro aced the task :-)

But maybe its a config issue.

[mark_l_watson]

That sounds correct: Pro for longer agentic tasks, Flash is fine for writing short programs, finding things for me in a large code base, etc.

[wolttam]

Have to ask: did you try 'xhigh' thinking effort with Flash? I also found it nearly unusable on just 'high', but on 'xhigh' it's nearly equivalent to Pro's 'high'.

[wolttam]

I found Flash to be a bit shaky as well until I started using it in xhigh/max thinking effort, then it became my daily driver. It runs quite well on a couple of DGX Sparks.

I still wish it was a little better, but there's hope for another model checkpoint (maybe with some of GLM 5.2's goodness distilled into it, that would be nice).

[girvo]

> I found Flash to be a bit shaky as well until I started using it in xhigh/max thinking effort

This is true for most of the open weight chinese models, to be fair. They're really built around long reasoning chains.

Also you're making me want a second Spark-alike :') but they're so expensive...