[feintruled]

This is very cool. I wonder if we are going to have to get used to a new paradigm in software, where you have tools that are incredibly powerful and that you just accept that sometimes it 'gets it wrong'. There's no debugging, no root cause analysis, just a shrug of the shoulders and 'sometimes it gets it wrong mate, what're you gonna do?'. This is probably the mental model most laypeople have of software already I suspect but for software engineers it's somewhat of a shift. Bit of a deal with the devil, perhaps.

[reaperducer]

tools that are incredibly powerful and that you just accept that sometimes it 'gets it wrong'. There's no debugging, no root cause analysis, just a shrug of the shoulders and 'sometimes it gets it wrong mate, what're you gonna do?'.

So, pretty much what we have now with the vast majority of mega-tech companies with zero customer service. Plus all the growth-hack startups playing "monkey see, monkey do."

[Eddy_Viscosity2]

> you have tools that are incredibly powerful and that you just accept that sometimes it 'gets it wrong'

I think this is already the case with existing software tools and developers. That's how code ships with bugs both known and unknown.

[Ferret7446]

The difference is that you can throw a senior engineer at a bug and know that the issue can be root caused and fixed because the behavior is "fundamentally deterministic", whereas with AI for the foreseeable future all you can do is maybe tweak the model and pray.

[diggan]

> There's no debugging, no root cause analysis, just a shrug of the shoulders and 'sometimes it gets it wrong mate, what're you gonna do?'

This been the case for as long as I can remember, seems to have more to do with individual developers typical methodology rather than the tools available.

I remember a bunch of issues with early npm versions were resolved by deleting the node_modules directory and running `npm install` again. Sometimes it borked the directory, sometimes it didn't, deleting everything and beginning from the beginning resolved many of those issues.

[feintruled]

You are absolutely right of course, most day to day bugs we don't have either the inclination, time or knowledge to root cause much less fix, but I feel it's a comfort to know that you (or someone) could.

[GTP]

Well, as in many cases, I think it depends. There are some situations where you can accept errors, but in some others definitely not: imagine you're trying to delete a set of files from a directory that contains other files you're interested in. If the AI commits a mistake and deletes some of the other files you will be disappointed. Now, you should have a backup. But what if you had the AI assistant come up with the backup command for you and by mistake it didn't include that directory?

[feintruled]

I wonder how we might design for such a system. I'd say as a starting point any action should be undo-able. Then you give it a go to see if it works, and if it didn't you can always get back. I've read this is good practice in any system, as a user can get inured to 'are you sure' dialogs and just click through them reflexively.