[seanmcdirmid]

If LLMs can ever produce implementation and tests opaquely, then perhaps we can instruct at the level of requirements. They will never be good enough at the beginning and we will just provide feedback over the working prototype the LLM generates and demonstrates?

I haven’t seen technology move this fast before, so I wouldn’t make any hard predictions about how long actual code written by humans survives. We don’t really need AGI at this point to have opaque coding solutions, even if the LLMs should still be better.

[daxfohl]

I won't say "never", but I guess I left out the caveat that I tried this for a hack week demo once. Working in compliance and onboarding, there's a whole lot of logic around what you have to collect and check, what you have to do around user experience, what steps you can fall back to if something fails, etc. I tried to take all our code and translate it to English to feed to an LLM and see what it would do. But the final result was that, even if it worked perfectly, the English description was longer and more convoluted than the code, would've been harder to make changes to, yada yada, so even if it worked perfectly, it would be a worse option than code.

But maybe there's a future where a "best of both worlds" that intermingles structured code with unstructured verbal instructions, so that you can ensure that the important aspects of your requirements are explicit and deterministic, and the filler parts can be in English descriptions. It'd compile, and you get red squigglies in your editor if something doesn't make sense, maybe even type hints on the English somehow. I think that'd be a pretty good "best of both worlds" because it'd really let you separate the signal from the noise, which is the problem you have when using either English or programming languages distinctly.

[daxfohl]

Though now that I say that, I wonder if maybe the approach should be, instead of specifying the onboarding procedure in terms of steps, maybe it would be better to specify a goal, the constraints, and some optimization function, and allow the AI to figure out what the workflow steps should be. So it would act higher level, more as a constraint solver than a code generator. That could be easier to interact with, allow constraints to be read and updated more easily, and not require diving much into code at all.

Hmm, I don't think LLMs are quite there yet, but I could see this being a potential way to do things, that may fit better with their strengths, and allow more to be done without digging into actual code.

Now I just need to dust off my old Prolog books and figure out how to use coding assistants in a similar fashion.

[seanmcdirmid]

I think the magic key might be LLMs that can build and test code, and then revise it and test it again, revise tests, etc...all while accepting feedback from human users about what the requirements really are (because they aren't going to be complete at the beginning). We might already be there with LLMs and we just haven't figured out the workflow yet (it isn't just one LLM activation, for sure, it has to be a series, with feedback from test execution and reviews...like real SWEs).

[daxfohl]

Yes. I tried the prolog style requirements last night too and it wasn't great either. Some things are easier to say in "rules" and some things are easier to say in "steps". Trying to get all the way to "rules" ends up sounding like Yoda talk in places. Where with "steps" you feel the need for a GOTO, with "all rules" you feel the need for the infamous COMEFROM keyword in places.

So, yeah, I think what you say is the only logical conclusion. Waterfall development doesn't work well for humans, and it's probably not going to work for AI either. In fact, spending a ton of time formalizing a huge requirements doc is going to be even more of an antipattern with AI. When you can just have a conversation, hammer out details, identify nuances, plan for the future, all while seeing the feature in real time, that's going to be a million times more productive. The requirements doc can come with a full demo and a green button to launch it into production immediately after (or during) the review meeting.

So yeah, we'll probably never have "English as code", but rather an iterative process that leads to both English and code descriptions of the feature.

As far as DSLs, I think they still fit. If you see a pattern that makes sense to encode in a DSL, the LLM will happily do that, and doing so could help the AI optimize its limited context and make the invariants of the domain much more explicit.

Fun times.