[bjourne]

LLMs work by generating the most likely continuation to a prompt. But they can also generate multiple likely continuations. This create multiple branches which in turn can generate even more branches. The LLM can then evaluate the branches, prune the unpromising ones, and merge the best ones. More branches means more tokens, means more effort.

[simianwords]

this has nothing to do with the thinking effort however

[bjourne]

Yes, it does. Breadth of search is exactly what the effort setting controls.

[pyentropy]

LLM-judge/parallel branching ≠ multi-token prediction ≠ reasoning effort.

See https://developers.openai.com/cookbook/articles/openai-harmo... and src/openai/types/shared/reasoning_effort.py

[bjourne]

[flagged]

[simianwords]

No it doesn't and lets not call people names. You can verify this using ChatGPT or anything else. You are mistaken and there are no "branches" happening.

[bjourne]

[flagged]

[FergusArgyll]

I think you may be confusing the openai "pro" series models with thinking. Thos are rumored to be multi "branched"