[theptip]

Great eval! If you are comparing search strategies, ultra mode is likely superior. Would love to see a follow-up evaluating that.

Ultra can fan out parallel investigators, run adversarial review at defined checkpoints, and do a bunch of other smart stuff to avoid getting stuck in a local optimum.

Generally as the OP notes, /goal works better for single-track investigations or small scale scatter/gather.

[ttul]

I think ultra mode needs to be more clearly documented (or perhaps cautioned against!). Most devs - myself included - who saw “ultra” mode figured that it’s just a magic bullet that makes the model work harder and achieve better results. But, for many tasks, ultra mode is possibly worse and certainly more expensive.

[theptip]

Yeah I think having a detailed mental model of what it’s actually doing helps.

At least for Fable/Opus (didn’t confirm for Sol yet) Ultra means “write an ephemeral programmatic harness encoding this workflow”. There is actually a TS harness that gets run for the workflow.

If you have a task where the agent/sub-agent pattern works, Ultra just adds indirection.

I think it is possible to get an intuition for an individual model but really you need to eval to be sure. My heuristic though is if you need to treat each work item differently depending on the results, probably agent/subagent. If you want to do the same steps across some queue / tree / DAG of work items, ultra is a better bet. (Or actually write a durable scaffold if you are going to repeatedly run it over > thousands of items.)

[embedding-shape]

> that it’s just a magic bullet that makes the model work harder and achieve better results

"Ultra" is a harness feature, and has nothing to do with the model itself. If OpenAI wanted to, they could offer "Ultra" with any of the GPT models, although 5.6 supposedly been trained with this specific harness feature in mind. "max" is basically the top "work harder and achieve better results" parameter for the reasoning effort for the current models.

I had good success with it initially, until I discovered that OpenAI encrypts the prompts that the main model sends to the sub-agents (even in Codex, in local files you only see cipher text), then I completely dropped any experimentation of it as without introspection, it becomes basically useless for any real usage.

[ttul]

Oh yes, I know it’s a harness feature.

[alexpotato]

> But, for many tasks, ultra mode is possibly worse and certainly more expensive.

Went through something similar. Fable would just spends minutes thinking, processing, confabulating etc.

I dropped down to Haiku and got an answer in >30 seconds.

[Huppie]

Simon Willison recently had a useful tip instructing Claude to use its judgement to pick an appropriate model for tasks.

After a short discussion about the idea with Claude mostly on how it fits in my workflow and what models / effort I would like for certain tasks it placed a paragraph in my global Claude.md and it has worked wonders. Ultra became a lot better (faster, cheaper for the same output) and the amount of time Fable gets stuck overthinking things are reduced to the places where I think that model makes sense, for the rest it started fanning out a lot to Opus, Sonnet and even Haiku.

[kingkongjaffa]

How does it change model mid conversation?

Does it spawn sub agents with different models or is it the same single conversation dynamically switching models?

[Huppie]

Subagents indeed. This [0] was the post I was referencing, and here's the original suggested prompt:

> For all coding tasks use your judgement to decide an appropriate lower power model and run that in a subagent

As said, I tweaked it a little bit to my preference (mostly because it started delegating a bit too much) and had it placed in my global claude.md but that's the gist.

[0] https://simonwillison.net/2026/Jul/3/judgement/

[theptip]

Yes, sub-agents I believe. You need to pay to reload context if you switch the main conversation model effort.

[lucas_t_a]

i let it run once to just generally find any bugs using all my quota, them once i get back to it i ran on effort /max to continue with explicit don't run workflows fix all yourself, it was ultra effective.

ultra becomes a mode where your model enhances your prompt them it is execution multiplied by all other agents, which have them going in all sorts of directions.