[tuchsen]

Hey, I started tinkering with this last night, your comments probably saved me a lot of time.

It's more work, but maybe language specific tooling as a first pass? I'm wondering how far you'd get by feeding it all the type information first (from lets say rustdoc as a specific example), and then asking the LLM to understand the structure of the program.

Then taking that output (which you could cache) + any source file local context information + the users request for a change.

[anotherpaulg]

Ya, I played with giving a ReAct loop access to some python `jedi` tools for navigating a python code base. Considered wiring up language server protocol (LSP) into the ReAct toolchain as well, but couldn't find easy bindings for that.

There's 2 pieces to this puzzle:

1. Condensing the code based to fit into the context window.

2. Getting GPT to generate good code modifications.

A big stumbling block I have encountered with all of these approaches is that when you feed GPT condensed code it tends to GENERATE similarly condensed code. It doesn't fill in the details for the new code it's supposed to be writing. Rather it just generates stubs and comments like it was shown.

[tuchsen]

Wait, you cant just reinsert the condensed code into the original source file and then have it expand it? ChatGPT will already expand (or try to) a comment into an implementation for me. I'm sorry if I'm missing something obvious here, your much further along on this than I am.

[tuchsen]

As an aside, I'd love to see the code you've got so far. How did you wire up the ReAct loop? Haystack, Langchain, or just direct API calls to OpenAI? Getting LLM's to talk to the LSP protocol directly seems like a good idea as well.

[anotherpaulg]

I've been mostly using langchain and the direct openai api.

None of my code for this is worth sharing. It's all quick and dirty early experiments just to test if/how GPT handles various approaches.