And that's just context. It also has to be capable of debugging its own code, and looking for resources when it doesn't have the knowledge to come up with a solution.
That's a good point, and I guess the meta point is that there are a thousand and one things I haven't mentioned here that it would have to do as well. To some extent perhaps we could solve this from the LLM the same way a human would: by iteratively plugging in the errors generated by the output back into the bot and taking some next step based on the suggestions generated.
But then we'd have to coerce the bot into generating structured responses that can act as next steps.
It doesn't have to do any of that - if an AI assistant enables a team of 3 developers to do what just last year needed 6 programmers, then it has automated away half of the jobs, even if those 3 people are the only ones who can debug the code and look for resources when needed.
But then we'd have to coerce the bot into generating structured responses that can act as next steps.