[hegem0n]

I want to see the evolution of context management tools for coding with GPT.

- implement a function, providing structural context (e.g. model, service, controller) with stubbed modules, but leave out utilities/libraries and database model - drop the structural context, and ask it to expand the stub implementation with some additional context about the database model - drop the database model context, and ask it to refactor the solution with some additional context about utilities/libraries

I think this is doable, if we can build up some utilities for context management and iterative development, GPT should start to be usable on large code bases. It could work similarly to how one person wrote an entire novel using GPT.

[anotherpaulg]

I have been experimenting with exactly this. I hit the context window limit while developing a small web app [1] by having GPT do all the coding.

I've tried a few things:

1. Summarize/collapse the code.

  - Use GPT to summarize code blocks into a 1 line comment.
  - Collapse all the top level code blocks into their summary line. This turns the entire codebase into a much smaller "top level map".
  - Use a ReAct pattern via langchain to have the LLM itself try and determine which collapsed blocks need to be recursively "expanded" to understand the code with respect to the user requested feature/bugfix/change/modification.
  - Feed GPT this partially expanded code base along with the user requested change.
  - Have it spit back the modified version of that partially expanded code.
  - Apply the GPT changes back to the original source files.

2. Explicitly ask GPT "which parts of the code are relevant to this change request" or "which parts of this code would need to be modified to make the needed changes", etc.

3. Again using ReAct, give GPT tools to "grep" and "cat" the code so that it can explore the codebase to find and understand relevant chunks. I've even armed it with bash in a sandbox. It started importing and running parts of the python code I was operating over.

None of these approaches have panned out fully yet. But there are promising signs.

[1] This is the small webapp that GPT wrote for me. I'm working on it mainly as a forcing function to explore these sorts of "GPT as junior developer / coding collaborator" workflows.

https://github.com/paul-gauthier/easy-chat#created-by-chatgp...

[tuchsen]

Hey, I started tinkering with this last night, your comments probably saved me a lot of time.

It's more work, but maybe language specific tooling as a first pass? I'm wondering how far you'd get by feeding it all the type information first (from lets say rustdoc as a specific example), and then asking the LLM to understand the structure of the program.

Then taking that output (which you could cache) + any source file local context information + the users request for a change.

[anotherpaulg]

Ya, I played with giving a ReAct loop access to some python `jedi` tools for navigating a python code base. Considered wiring up language server protocol (LSP) into the ReAct toolchain as well, but couldn't find easy bindings for that.

There's 2 pieces to this puzzle:

1. Condensing the code based to fit into the context window.

2. Getting GPT to generate good code modifications.

A big stumbling block I have encountered with all of these approaches is that when you feed GPT condensed code it tends to GENERATE similarly condensed code. It doesn't fill in the details for the new code it's supposed to be writing. Rather it just generates stubs and comments like it was shown.

[tuchsen]

Wait, you cant just reinsert the condensed code into the original source file and then have it expand it? ChatGPT will already expand (or try to) a comment into an implementation for me. I'm sorry if I'm missing something obvious here, your much further along on this than I am.

[tuchsen]

As an aside, I'd love to see the code you've got so far. How did you wire up the ReAct loop? Haystack, Langchain, or just direct API calls to OpenAI? Getting LLM's to talk to the LSP protocol directly seems like a good idea as well.

[anotherpaulg]

I've been mostly using langchain and the direct openai api.

None of my code for this is worth sharing. It's all quick and dirty early experiments just to test if/how GPT handles various approaches.

[akiselev]

Have you tried passing types, function definitions, etc. into the OpenAI embeddings API? It might be possible to automatically build up the context using embedding vectors generated between the request prompt and the code.