[kgeist]

Judging by the examples, if I understand it correctly, J-space supports higher-order logical / multihop transformations, but it is limited in size because of the limited network depth (max number of layers). When we emulate "reasoning," we basically extend J-space and allow the higher-order transformations to continue for longer, toward a more logical conclusion.

It sounds like instead of generating reasoning tokens end-to-end, we could probably only loop the middle layers (the ones most related to J-space) while skipping the first and last layers (less related to J-space) It probably explains why [0] worked. OP accidentally extended J-space? Also reminds of looped transformers.

[0] https://news.ycombinator.com/item?id=47431671

[tedd4u]

See also Ouro [1]. Good citations in the “related work” section.

[1] https://ouro-llm.github.io/