[intalentive]

Overall I really like these transformer RNNs. They are basically EBMs learning an energy landscape that falls into a solution, relaxing a discrete problem into a smooth convex one. Reminds me of other iterative methods like neural cellular automata and flow matching / diffusion. This method looks promising for control problems: just tumble your way down the state space, where each step is constrained to be a valid action.