[jtmarmon]

I just had it fix some outdated copy in a part of the UI. The nice thing is I didn't have to find the file myself, I just described what was wrong like I would a junior eng and let it find and fix it. Worked on the first try!

[williamzeng0]

That's exactly the use case we want. We also let you specify the file path (ex: "main.py").

We noticed that Sweep's search works way better if there are comments, because the comments match up really well with the search queries (language <-> language is easier than language <-> code)

[applgo443]

May be rewriting user's description might help you match code better?

Similar to the prompt engineering for previous era GPT completion models.

[williamzeng0]

We do end up doing a GPT based rewrite. The initial description is really valuable too though, and we want to keep that throughout the workflow. It's kind of similar to a spelling correction or query intent system. If it's high confidence you can override their query, but ideally you use the original one too.

[applgo443]

Did you consider first asking LLM to explain what a code snippet does and use that instead?

It'd significantly increase the costs though.

[williamzeng0]

I didn't mention this point, but we actually do that during the modification. We ask the LLM to extract the necessary subcontext from the main context. It doesn't increase the costs much, but it does help performance because the unnecessary context is stripped away.