[ai4ever]

i dont like to discourage or be a naysayer. but,

dont build a platform for software on something inherently unreliable. if there is one lesson i have learnt, it is that, systems and abstractions are built on interfaces which are reliable and deterministic.

focus on llm usecases where accuracy is not paramount - there are tons of them. ocr, summarization, reporting, recommendations.

[viraptor]

People are already unreliable and non-deterministic. Looking at that aspect, we're not losing anything.

[ben_w]

As a result of human unreliability, we had to invent bureaucracy and qualifications for society at large, and design patterns and automated testing for software engineers in particular.

I have a suspicion that there's a "best design pattern" and "best architecture" for getting the most out of existing LLMs (and some equivalents for non-software usage of LLMs and also non-LLM AI), but I'm not sure it's worth the trouble to find out what that is rather than just wait for AI models to get better.

[ai4ever]

people may be unreliable but the software they produce needs to work reliably.

software system is like legos. they form a system of dependencies. each component in the chain has interfaces which other components depend on. 99% reliability doesnt cut it for software components.

[viraptor]

I'm not sure, but you may be misunderstanding the project, or trying to make some point in missing. This project just automates some code tasks. The developer is still responsible for the design / reliability / component interfaces. If you see the result doesn't match the expectations, you can either finish it yourself, or send this tool for another loop with new instructions.

[ai4ever]

let me test it out, and then provide better feedback.

[falcor84]

>the software they produce needs to work reliably

The word "need" is an extreme overstatement here. The vast majority of software out there is unreliable. If anything, I believe it is AI that can finally bring formally verified software into the industry, because us regular human devs definitely aren't doing that.

[ai4ever]

thats a fair statement to say that humans cannot be the gatekeepers for accuracy or reliability.

but why should the solution involve AI (thats just the latest bandwagon)? formal verification of software has a long history which has nothing to do with AI.

[cma]

Probably because of Google's recent math olympiad results using AI-directed search in formal proof systems.

[stale2002]

> but why should the solution involve AI

Because AI is able to produce lots of results, covering a wide range of domains, and it can do so cheaply.

Sure, there are so quality issues. But that is the case for most software.

[dartos]

What part of “AI” implies “formally verified?”

[ekianjo]

And that's precisely why we don't use people to do tests and to ensure that things work reliably. We use code instead.

[ben_w]

I've had trouble trying to convince a few different people of this over the years.

One case, the other dev refused to allow a commit (fine) because some function had known flaws and was should no longer be used for new code (good reason), this fact wasn't documented anywhere (raising flags) so I tried to add a deprecation tag as well as changing the thing, they refused to allow any deprecation tags "because committed code should not generate warnings" (putting the cart before the horse) — and even refused accept that such a warning might be a useful thing for anyone. So, they became a human compiler in the mode of all-warnings-are-errors… but only they knew what the warnings were because they refused to allow them to be entered into code. No sense of irony. And of course, they didn't like it when someone else approved a commit before they could get in and say "no, because ${thing nobody else knew}".

A different case, years after Apple had switched ObjC to use ARC, the other dev was refusing to update despite the semi-automated tool Apple provided to help with the ARC transition. The C++ parts of their codebase were even worse, as they didn't know anything about smart pointers and were using raw pointers, new, delete everywhere — I still don't count myself as a C++ despite having occasionally used it in a few workplaces, and yet I knew about it even then.

And, I'm sure like everyone here has experience of, I've seen a few too many places that rely on manual testing.

[viraptor]

That's not universal. QA teams exist for things which are not easy to automatically test. We also continuously test subjective areas like "does this website look good".

[ekianjo]

Agree. but the boundaries of automation are progressing year after year. We wont be able to replace everything humans do anytime soon for testing but still a lot can and will be done.

[usrbinbash]

Yes, they are, and that's precisely why we use computers and deterministic code for many tasks instead of people.

[elicksaur]

I really don’t like the denigration of humanity to sell these products. The current state of LLMs is so far away on “reliability” than the average human that these marketing lines are insulting.

It really seems like the tech-bro space hates humans so much that their motivation in working on these products is replacing them to never have to work with a human again.

[djeastm]

>I really don’t like the denigration of humanity to sell these products.

Sure, but then humanity was denigrated the first time a calculator was used to compute a sum instead of asking John Q Human to do it.

I'd argue that the more we find ways to replace humans with AI, we're more clearly defining what humanity is. Not about denigration or elevation, just truth.

[atemerev]

> systems and abstractions are built on interfaces which are reliable and deterministic.

Are you sure we live in the same world? The world where there is Crowdstrike and a new zero day every week?

Software engineering is beautifully chaotic, I like it like that.