Hacker News new | ask | show | jobs
by inimino 2928 days ago
> The system did exactly what it was intended to do, it was the humans who screwed up.

It was the humans who designed and who chose to deploy a system without a human in the loop, and without an override (even after a director was involved) that screwed up.
1 comments
krapp 2928 days ago
Maybe, but the humans who didn't renew his employment status knew how the system worked. They screwed up more.
link
inimino 2928 days ago
They screwed up, but that happens. The point where the system takes over and even the higher-ups can't override it is where the story becomes Kafka-esque.

If you build an automation system that goes out of human control after a human error, that is a failed design.

link
krapp 2927 days ago
>If you build an automation system that goes out of human control after a human error, that is a failed design.

Unless it was designed with the intent that human intervention should be impossible once a process was started. That would make it a very poor design, but not a failed one.

I've seen lots of internal software that doesn't have failsafes or rollbacks - the operator is simply trained to follow procedure and then is expected to follow it. Software that considers operator error is more complex, and therefore more expensive, to produce. Cost often supersedes quality or flexibility when these systems are developed.

link
inimino 2927 days ago
"Self-destruct sequence initiated, cannot abort." should be left in the world of sci-fi.

If there is a physical process that cannot be stopped (rocket, nuclear reactor, oil well) then of course the system must be designed around that physical fact.

But this is an HR process. The "moving parts" are people.

If it was designed with the intent that human intervention should be impossible, the design was a failure.

Perhaps more likely it was just a failure of implementation (there's a cancel button but nobody knows where) or of imagination (nobody thought about whether the process could be canceled or not).

What if instead of HR, this was a financial process, playing out over days or weeks, entirely beyond the company's control. Would any sane CFO approve such a thing?

> Cost often supersedes quality or flexibility

That helps to explain it, but it doesn't excuse it.

link