[anotherpaulg]

Very cool project. How reliable are you finding your prompts? They look like good choices based on my experience prompting GPT-3.5 and 4 for code editing.

FYI, I think my open source tool aider would work out of the box to serve this use case. You would just run:

  aider file.py —msg “implement the comments”

Of course aider works with any popular language, not just python. And it can do a lot of other coding tasks. It's like pair programming with an AI.

https://github.com/paul-gauthier/aider

[chime]

I absolutely love aider and tell everyone I run into about it. Keep it the great work!

Question: can aider work with Ooba/llama.CPP/meta code llama on local llm? If not yet, are you planning on it?

So many users just can’t use GPT4/Copilot because of corporate policy. But they have Macs with M2.

[anotherpaulg]

Aider provides experimental support for LLMs other than OpenAI's GPT-3.5 and GPT-4. The support is currently only experimental for two reasons:

1. GPT-3.5 is just barely capable of editing code to provide aider's interactive "pair programming" style workflow. None of the other models seem to be as capable as GPT-3.5 yet.

2. Just "hooking up" aider to a new model by connecting to its API is almost certainly not enough to get it working in a useful way. Getting aider working well with GPT-3.5 and GPT-4 was a significant undertaking, involving specific code editing prompts and backends for each model and extensive benchmarking [0]. Officially supporting each new LLM will probably require a similar effort to tailor the prompts and editing backends.

Numerous users have done experiments with numerous models. None of these experiments have yet identified other models that look like they are capable of working well with aider. Claude has been the most promising so far, and the new Code Llama looks very interesting on first glance.

Once we see signs that a particular model is capable of code editing, it would be reasonable for aider to attempt to officially support such a model. Until then, aider will simply maintain experimental support for using alternative models.

There is more information on connecting aider to other models, local models and Azure models in the FAQ [1]. There are also ongoing discussions about LLM integrations in the aider discord [2]:

[0] https://aider.chat/docs/benchmarks.html

[1] https://aider.chat/docs/faq.html#can-i-use-aider-with-other-...

[2] https://discord.com/channels/1131200896827654144/11330607806...

[Terretta]

Paul's work on this problem space is worth following, or at least reading the thoughts and iterative engineering results such as this comparison of the GPT models and the new functions API:

https://aider.chat/docs/benchmarks.html