[emporas]

The process of decomposing the task into smaller steps and generate each step independently seems to be the correct way in my experience too. It works very well with GPT (chatGPT or GPT4).

>100k tokens is plenty.

The context window can be really helpful, in case there is a release of a new library and the user wants to generate code targeting the API of the library. When the training date stops at August 2023, any library released after that date is not known to the engine.

My general opinion in regards to context window, is that 1 trillion tokens context window still may not be enough for all use cases.