|
|
|
|
|
|
by rayxi271828
785 days ago
|
|
|
While it's clear that AI coding assistants still have a long way to go in general, I wonder how yet another startup, presumably based on the same technology at the back, is going to differentiate itself in a material way? There are only so many axes along which improvements can be made in this domain, aren't there? What are the bottlenecks that, if solved, will produce a true breakthrough, exactly? Doesn't the current approach have an upper limit that's inherent in the whole architecture, nay, even the whole foundational theoretical aspect of it? Would love to hear from anyone who has come across one AI coding assistant that's obviously head and shoulders above everything else. I've tried Copilot, CodeWhisperer, and Ghostwriter. |
|
|
• It runs locally.
• It is extremely fast.
• It is not overly aggressive. It only shows up sometimes and when it does I nearly always accept its suggestion.
• It's a native part of the IDE so doesn't interfere with autocompletion.
I gotta say, JetBrains nailed it with this one. Single line mode is a much better way to think about AI driven autocomplete. That said, it's also obviously much more limited. It helps and is pleasant, but isn't revolutionary.
The other AI assistant I use sometimes is aider.chat, it's an open source tool. You type what you want and it generates git commits for the requested change. This is clearly the direction to go in long term and is much more revolutionary, but there are still problems with it.
> What are the bottlenecks that, if solved, will produce a true breakthrough, exactly?
There's a lot of "low hanging fruit" here (not that low), because the big AI labs aren't focusing on code gen AI right now. I can think of at least 4 or 5 paper-sized research directions to make improvements. The challenge is that right now the only models any good at coding are GPT-4/Opus level models, so doing anything with them is impossible unless you're working at a tiny number of labs. I don't work there so the "obvious" ideas I have aren't of much use. That leaves explorations of whether you can take a great open source model and boost its coding skills a lot. I think you probably can, especially if you have the resources to do a Mixtral or a Llama 3 yourself. Most smaller labs seem to be focussing on general performance optimizations rather than trying to reach GPT-4 level skills so the number of labs capable of doing these experiments should go up with time.
From talking to people there's also a general fear I think that there's not much point working on some sorts of ideas because what if OpenAI come out with GPT-5 and everyone gets taught another Bitter Lesson (http://www.incompleteideas.net/IncIdeas/BitterLesson.html)? Better to just wait things out and see where model capabilities stabilize. From all the noise about licensing deals it's starting to sound like we're data constrained even for the companies that are pushing the boundaries of what fair use means, so maybe we'll see appetite to do smarter things with coding models next year.