[ttul]

This is what Codex is doing. The LM has been trained to work well with the kinds of tools that a solid developer would use to navigate and search around a code repository and then to reason about what it finds. It’s also really competent at breaking down a task into steps. But I think the real magic - watching this thing for at least 40 of the last 50 working hours - is how it uses command line tools to dig through code quickly and accurately.

It’s not relying on the LM context much. You can generally code away for an hour before you run out of context and have to run a compression step or just start fresh.