[activatedgeek]

I want to point out a tweet [1] that is very relevant to the miracle of CoT, and probably a simpler explanation.

  > Let's think "step by step"!

  > Another tidbit I like about data and prompts that miraculously work.
  > Searching for this phrase resulted in this website (among others),  
  > http://geteasysolution.com, containing many math step-by-step solutions. 
  > How common are they? Quite.

  > Makes you think.

[1]: https://twitter.com/yanaiela/status/1765077404043952516

[FeepingCreature]

Though that justifies the specific phrase, it doesn't really contradict the usual explanations of how CoT works. Like... the phrase directs it into the conceptual space of a website that has lots of CoT examples, but if CoT didn't help it think, that wouldn't actually result in better outputs.

[activatedgeek]

I hesitate to the use description as "think," just biasing correlations for subsequent generations.

In any case, there is at least one work that shows that CoT may not be necessary and biasing the decoding path via logit probabilities is also promising. [1]

One could argue it still doesn't contradict the benefits of CoT, but I suspect there is nothing fundamental about CoT, except that we happened to have been pre-training on sequences that use certain prompts that were easy to conceive from a human's perspective.

[1]: https://arxiv.org/abs/2402.10200

[FeepingCreature]

Absolutely. QuietSTaR is going to make CoT obsolete. It's just, it won't make it obsolete by showing that CoT does nothing, but by getting the LLM to embed invisible CoT-like token sequences into the context on its own. That's a victory for CoT, not a loss.

"Let's think step by step" is a hack. It was always a hack. Its main payoff was showing people where the model had a weakness and how to (hackily, heavily dependent on training data and phrasing) route around it. Now with QS, the models will be able to bypass that weakness on their own.

> I hesitate to the use description as "think," just biasing correlations for subsequent generations.

This is of course a fully general description of any iterative computation. :)

[activatedgeek]

Makes sense. Thank you for a the QS reference!