Humans aren’t any better. That’s why we have OSHA etc. I think you’re hoping for a formal logic based AI and I’ll wager no such thing will ever exist - and if it do, it would try to kill us all.
People have fairly consistent faults. LLMs are nondeterministic even in terms of how they fail. A high value human resource can be counted on to deliver. That, imho, is in fact one of the primary roles of good management: putting the right person in the appropriate position.
Process engineering has worked to date because both the human and mechanical components of a system fail in predictable ways and we can try to remedy that. This is the golden bug of the current crop of "AI".
People make bad decisions all the time. Insanity is not required. My remark was pointing out the larger failure mode of people acting contrary to the good of the team for personal gain, eg creating a problem and blaming someone else to reduce their chances of competing for a promotion. But to your point, an SDE doesn’t need to be insane to bypass a 2 PR and force a change into production. They just need to be panicked, or overconfident, or overworked.
> They just need to be panicked, or overconfident, or overworked.
One of the best thing about digital computers, compared to humans, was that they can't be the first or the third thing you mentioned; unfortunately, they absolutely are the second ("the machine does exactly what you told it to do, not what you want it to do"), and at inhuman speeds. Presumably, AI would (need, actually — Nick Bostrom puts a fairly reasonable argument for that in his "Superintelligence") fix that second bullet point, and then everything will be peachy.
Instead, we have people on the internet arguing that it's not a problem, since people too have this same problem. Which is a problem. But not a problem. Ugh.
Computers absolutely can be overworked. Plenty of outages caused by system overloads. Or a system deletes a file because it believes it to be no longer in use but only because some queue was full. I’m not arguing that it’s not a problem because humans have the same problem. Part of my job is making sure humans can’t fuck it up either. I’m saying “assume the worst” and make sure the processes catch human and AI mistakes.
Also, I think Nick makes the same point as me: AI will attempt to kill us.
Formal logic AI systems have existed and were popular in the 1980s. One of the problems is that they don't work - in the real world there are no firm facts, everything is squishy, and when you try to build a large system you end up making tons of exceptions for special cases until it becomes completely untenable.
Non-deterministic systems that work probabilistically are just superior in function to that, even if it makes us all deeply uncomfortable.
I don't know what definition of AI you're using, but plenty of ML algorithms operate deterministically, let alone most other logic programmed into a computer. I don't see how your statement can be right given that these other software systems also operate in the real world.
We're different.
People have fairly consistent faults. LLMs are nondeterministic even in terms of how they fail. A high value human resource can be counted on to deliver. That, imho, is in fact one of the primary roles of good management: putting the right person in the appropriate position.
Process engineering has worked to date because both the human and mechanical components of a system fail in predictable ways and we can try to remedy that. This is the golden bug of the current crop of "AI".