[trzy]

This is very cool! And I think it's a perfectly good approach. My understanding is that the usual way of doing things with LLMs is to train them on a series of specialized tokens that represent actions in your environment. E.g., this sequence of words results in this completion of action tokens.

The code approach is intriguing and I'd like to explore further but controllability is a real problem and bullet-proofing it would require a lot of effort, if it's even possible at all. I do think that a hybrid environment where one speaks what they want and then sees the code and can interact with it in a friendly way would be very intriguing for a sandbox experience.