[wallacoloo]

it could be interesting to gauge how entwined the “how did you arrive at that answer” process is with the answering itself. i.e. which paths do they share? even at this early a stage: is there some structure which is used to determine the operand(s) that’s leveraged in both of these prompts? is the “how did you X” answer leveraging most of the “X” circuitry and just bailing out early? or does it deviate as early as post-tokenization?

philosophers would like to know.

[jcims]

My initial guess is they have nothing to do with each other. It would be like explaining why the next idea pops in your head. You can create a rational explanation but there's no way to test it.

[wallacoloo]

my thoughts too, based on limited understanding of GPT. but the more pressure you apply towards compressing the neural network during training, the more circuitry these paths are likely to share. it would be interesting to see just how much and which parts could be folded together before you start to lose significant fidelity (though unfortunately the fidelity seems too low today to even try that).