|
|
|
|
|
|
by whimsicalism
643 days ago
|
|
|
No? > I think this submission paper is talking about reinforcement learning as part of/after the main training Reinforcement learning to promote a particular type of self-correction response > They might have done that for O1, but the bigger change is the "runtime train of thought" that once the model received the prompt and before giving a definitive answer, Also reinforcement learning to promote certain reasoning trace > o1 and this paper talk about using techniques to create a useful reward function to use in RL that doesn't rely on human feedback. Exactly -> the same thing |
|
|
I take this to mean during weight updates, e.g. training.
> "runtime train of thought"
I take runtime here to mean inference, not during RL. What does runtime mean to you?
Previous approaches [0] successfully used inference time chain of thought to improve model responses. That has nothing to do with RL though.
The grandparent is wrong about the paper. They are doing chain of thought responses during training and doing RL on that to update the weights, not just during inference/runtime.
[0] https://arxiv.org/abs/2201.11903