[esalman]

I think what the author is suggesting is that decision making, especially those which can critically affect a person, should not be automated by machines. There must be a mechanism in place which is open to scrutiny (preferably by a qualified human) and accountability.

Take the example of predicting terrorism threats based on facial queues of stress or fear. Machines lack context, which a qualified human would otherwise take into consideration. You can be stressed out because you might be accompanying a child, or fearful that you might miss a flight. If a TSA agent deports someone simply because a machine recommended it, that would be inhumane.

People like to argue that more regulation will adversely affect automation and/or growth/scaling of technologies and businesses. Growth is important but it must not cost us our humanity.

[Verdex]

Personally, I think this type of thing falls into what I like to call the "all problems are management problems" category.

If a TSA agent deports someone simply because the machine recommended it, then that is a problem. But if a TSA agent deports someone simply because they were having a bad day, then that is also a problem.

I guess I don't care what tools those in authority are using (their intuition or a mechanical intuition or a database lookup), but what I care about is whether or not innocent people without the ability to navigate an appeal system are being erroneously penalized.

And ultimately, it's the responsibility of management to make sure that the people/tools they assign are doing a good job. If they fail, then management needs to find new people/tools. If management fails to do that, then all the AI regulation in the world isn't going to do much good.

[esalman]

Well, the purpose of the AI regulation is specifically to hold the "management" accountable when they fail to prevent misuse of the technology. The regulations are to be enacted by the lawmakers. As I see it, the debates like the one we are having right now will determine how the regulation will be shaped eventually.

[Verdex]

Yes, but I'm worried about the people component of my "people/tool" equation. If management sees that costly fines occur when AI is used, then maybe they'll abandon it and just use people instead.

However, if they hire a bunch of power hungry sociopaths who are very good at hiding their malicious oppression and who also bring in donuts every Thursday to stay on their bosses good side, then the situation could easily lead to worse outcomes for the people who have to deal with this system.

If we create a computer system that oppresses 1% of innocent people, then that is a problem. However, I don't consider it a win to ban the computer system and replace it with a human system that oppresses 10% of innocent people. Like, the situation isn't better because humans are oppressing humans instead of a computer doing the oppression.

That's why I was focused on management. I don't care that things are going badly for some specific technology related reason. It's management's job to fix it regardless. If management can't rely on the technology for regulatory reasons, then they might rely on people who do just as bad of a job. And hey, that scenario is even better for management because if they hire a bad actor who gets caught then that person faces the consequences and not them.

[visarga]

Maybe we're going to develop a specialised court system for AI where humans can sue for AI-related injustice. I don't trust companies to self regulate effectively.

[harles]

We haven’t needed such courts for computers, the internet, or other algorithms. I fail to see how special courts are needed for AI. Our current legal system appears perfectly suited for AI - at least no worse than for other tech.

[visarga]

> decision making ... should not be automated by machines. There must be a mechanism in place which is open to scrutiny (preferably by a qualified human) and accountability

You're saying "I don't trust AI, I want human supervision", but this works both ways. Sometimes we don't trust the humans and would prefer a neutral AI. Humans do terrible things to other humans. Who's going to review my appeals? What are their biases? Are they any more trustworthy than a model?