[wokwokwok]

There are two parts to “writing a Devin”.

1) a pretty user interface, a backend service layer, all the normal things you’d need for a desktop app or saas.

2) a backend that can actually do work.

I’m going to go out a limb here and say, building an MVP of the UI is a waste of your time.

…because, unless you know how to build the backend (ie. part 2), you actually, have no idea what you need to build for part one.

You can copy of the UI from the Devin videos.

You can build your own langchain framework.

You can fine tune an open model on GitHub issues.

…but just like having building a gpt4 is harder than just “add more params”, building something that works like Devin appears to work requires a reasonably sharp “step up” in capability between what literally everyone has been doing with gpt4 until now, and being able to turn that into a useful framework for solving actual engineering tasks.

So… don’t hold your breath. If you see someone building a UI (like this https://github.com/OpenDevin/OpenDevin/tree/main/frontend, https://github.com/stitionai/devika/tree/main/ui; just read the commit log, it’s basically just ui) it means they’re doing the easy work (part 1) because they don’t know how to do the hard part (part 2).

…so, interesting, but this doesn’t smell like a really serious effort (at least yet).

I guess you could argue that it’s important “setup infrastructure” stuff that any project starts with… but I’m just sceptical.

I can draw pictures of a Devin too. A serious effort would be trying to replicate what Devin does not what it looks like.

[mellosouls]

So… don’t hold your breath. If you see someone building a UI (like this https://github.com/OpenDevin/OpenDevin/tree/main/frontend, https://github.com/stitionai/devika/tree/main/ui; just read the commit log, it’s basically just ui

Yes. If you link to the UI sections of a repo you will likely see "basically just ui" commit history.

I guess you could argue that it’s important “setup infrastructure” stuff that any project starts with… but I’m just sceptical.

Of course it's an early stage pilot, it's only been developed since the Devin hype, why should it compete in quality at this very early time?

[moffkalast]

Well I don't see why not set up the UI part first if that's what currently possible. There's no shortage of LLM frontends right now but practically all of them are very meh (if I see another overused default theme Gradio app I will stab my eyes out).

When Zuck releases Llama 3 in July or whenever we might have something to plug this into.

[CGamesPlay]

One reason why: they spent their launch hype on vaporware. If they had done the hard part and then launched with a default theme Gradio app around that, it would be much better for gathering excited developers.

[rbren]

OpenDevin dev here. I created the default agent.

We're very aware that we'll need great agents to be able to compete with Devin and others. We're currently setting up evaluation pipelines to evaluate various agents against SWE-bench.

Our thesis is that a community experimenting with various agents and agent architectures will outpace a private company on a single track. We're building the notion of an "agent hub" out of the gates--anyone can plug into the Agent interface and contribute their work. We're also discussing how to build a meta-agent, which farms out specific tasks to sub-agents.

It's early days though--we've only just gotten things wired together in a sort-of working demo. Stay tuned!

[NayamAmarshe]

Isn't Devin a GPT-4 wrapper too, with an increased context size?

[Garlef]

Isn't a web browser just a wrapper around http requests, with some memoization and rendering?

[NayamAmarshe]

Completely different things with different levels of complexities. The LLM is the one doing the heavy-lifting in the case of an agent. One does not need to build an LLM from scratch, but they do with a web-browser. With the LLM, the work is already 70% done, the rest is figuring out how to create an agent and fine-tune the params.

[jokethrowaway]

I don't get what's special about Devin. I noticed the same logic issues I've seen with ChatGPT or Copilot or Gemini.

OpenAI already did spend quite a bit on software development reinforcement learning and we already have models with huge context windows.

We've already seen OSS projects where ai agents collaborate to create code projects.

If you use CodeLlama, throw some engineering manpower to create feedback loop with compilers, linters and other tooling to improve the code generated you can probably match Devin's performance. I can get some pretty good results feeding back compiler's output to ChatGPT, if that were automated, my impression of ChatGPT would be much better.

So my bet is that an open source Devin is absolutely possible.

[wokwokwok]

> I don't get what's special about Devin.

Did you watch the YouTube videos?

I’m also skeptical it can do all the things it appears to be able to do, but I think it’s undeniable that what they show it doing (totally unverified as it is)…

…is significantly more capable than any other programming agent I’ve seen.

If openAI had that capability, they would be selling it.

They are not. I doubt very much that they’re just hoarding it and keeping it to themselves; gpt4 just can’t do it out of the box.

It is not that simple.

It is not just a fancy prompt for gpt4.

If it was that easy, someone would have done a year ago.

> If you use CodeLlama

Oh come on. These pathetic little open source models are shit compared the vendor offerings.

This is daydreaming about what might be, not what actually exists.

[mellosouls]

If it was that easy, someone would have done a year ago

Devin isn't as original as it's marketing claims, there are prior tools in this space (eg. Sweep). I've no idea whether it's significantly better than them, but certainly the magic is the underlying LLM, not Devin.

No reason at all why other devs - including open source - can't continue to provide competitors at that same surface level.

[paradite]

Yes. I can't believe the amount of negative comments on Devin or just brushing it off.

It is hard to build something like that.

If it's just a GPT wrapper, we would have tons of indie hackers building competitors and making 10k MRR by now.

The agents, prompts and orchestration are difficult problems to solve.

On the other hand, you have people thinking it's impossible to build something like that. I also disagree.

From a complexity perspective, coding a piece of software is no harder than driving an autonomous vehicle, since the logic is fixed and the environment is deterministic. If we can get to L5 for AV, I don't see why we can't get to L5 for coding.

[gerroo]

I wholeheartedly agree. I'm trying to get involved with this project and thought that it was an effort in the right direction. Appreciate the insights.

If they let me I'll get to work on the "hard part."