[_nalply]

Perhaps a good way would be to define a logics language for AIs or use an existing one like Prolog and let the LLM generate code then run it. It's a variant of giving LLMs access to some system and let them iterate till they find the solution.

The idea: When programming sometimes the solution is not exactly right but with feedback a better solution can be found. I once made that explicit to the LLM and I played the evaluator for the LLM and it seemed to work better. I am not surprised. Even humans usually don't just program on paper. In the early days of the computer science they had to do that. I experienced that myself: my first BASIC program was on paper and then when I had access, I typed it in. The experience is bad. I had to guess without feedback.

I can imagine that it doesn't solve all problems because not all is solvable by trying out things or the formulation of the problem doesn't describe the problem correctly. But it is better than not trying out things at all.

Please see Animats comment: https://news.ycombinator.com/item?id=41095417 and the answers there. They are saying more or less the same thing but more detailed and knowledgeable than me.