[dpark]

> must be able to express the same rigor in less words than existing source code

Yes but also no. Writing source means rigorously specifying the implementation itself in deep detail. Most of the time, the implementation does not need to be specified with this sort of rigor. Instead the observable behavior needs to be specified rigorously.

[tyleo]

That doesn't sound right. For example, there's plenty of software with the correct observable behavior which leaks credentials. So what needs to be captured goes beyond observable behavior.

[dpark]

Leaking credentials is observable behavior.

Certainly you could write specification for a piece of software, and the software could meet the specification while also leaking credentials. Obviously, that would be a problem. But at some point, this starts to feel artificial and silly. The same software could reformat your hard disk, right?

At some point, we aren’t discussing whether or not AI is doing a bad job writing software. We’re discussing whether or not it’s actively malicious.

[tyleo]

If leaking credentials is observable behavior. I don't get your point.

Memory leaks, deleting the hard drive, spending money would all be observable behavior.

By your reasoning that the "observable behavior needs to be specified rigorously" it seems like you'd have to list these all out. We do, after all, already have cases of AI deleting data.

That sounds harder and more error prone than what we're doing now by rigorously defining these defects out of existence in code.

[dpark]

My ultimate point is that source code specifies everything the software can or will do, but most of that is not part of the required specification. Most of it is implementation details you don’t care about.

The entire reason we have functions and components and modules etc is to isolate engineers from the things we do not need to care about. I should not need to care about the implementation details of most software, only if it meets my retirements.

The move to AI first software development will not happen because we find a way to specify as much in English as we previously would have specified in a programming language. The move will happen when and as we figure out how to specify the things that matter. We don’t need the same rigor. We need the correct rigor.

[skydhash]

> I should not need to care about the implementation details of most software, only if it meets my retirements.

The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details that have not made it into the specifications. One one side you have the platform and on the other side you have the interface contract (requirements). Saying what’s in the middle doesn’t matter is strange. Because both the platform and the interface are dynamic and can shift drastically from their 1.0 version.

[dpark]

> The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details

I disagree. Very often the reason the details don’t matter is that they are irrelevant. There are a million ways an app might remember my personal settings, as a simple example. SQLlite db, json file, ini, cloud storage, registry, etc. The specific implementation matters very little so long as it’s sane.

> Saying what’s in the middle doesn’t matter is strange

I understand your point but do not agree. I think over the next decade we will get increasingly good at specifying rigorously the parts of the surface that matter while increasingly caring less and less about the rest. We will not find a way to write rigorous code in English because that would necessarily be less efficient than just using a programming language.

[tyleo]

Yeah, I'm with you and I'm not seeing much sense in the argument above.

We should only need to specify observable behavior... but observable behavior is so broad that it includes common defects... but also we shouldn't need to specify the lack of defects even though they are observable.

It feels like, "the AI should read my mind and do the right thing without me fully specifying it."