[seveibar]

I have a tiny command line program called `dir-prompt`[1] to format and copy to clipboard all the code in a directory to something that can be pasted into ChatGPT or Typingmind, but it is a huge issue that the AI will be too brief on the generated code snippets.

Glide is much more structured. I gave it a real test on one of my repos and it outlined something I would expect from a junior engineer who messed up a lot of little details- e.g. function names and parameters were slightly off. I think if there was a type evaluation step/it was typescript-specific it would match an engineer with 3-9mo of experience (great!). But it failed to match basic codebase conventions e.g. using require on a codebase that is all imports.

For the authors, I would recommend using GPT4-turbo and huge context windows, or have an option to upload an OpenAI key. GPT3.5 is just not worth it- and I do sort of suspect GPT4 isn't being used because of the speed of the output but I'm not sure.

[1] https://github.com/seveibar/dir-prompt

[killerstorm]

Claude 3 is way better than GPT-4-turbo at actually doing stuff.

[ivanovm]

good to know, we'll give it a whirl. I tried it for like 30 mins in the anthropic web ui when it first came out and couldn't tell whether it was better right off the bat. Need to run more extended evals on that one and gemini 1.5

[ivanovm]

we use GPT-4-turbo across the board, 3.5 doesn't handle multi-step reasoning nearly well enough to be useful here.

We'll have to figure out how to extract codebase conventions efficiently and inject them into the coding prompts. Using up all 128k tokens has given us worse results (common theme), but hopefully with newer 1m context window models this concern away