[ling3]

Thinking through the consequences of this type of technology is important, but the narrative of "we can no longer allow technology X to go unregulated" as a vague catch-all for every negative externality is unhelpful. Regulate how? What concrete solutions is the author suggesting?

[visarga]

> Regulate how? What concrete solutions is the author suggesting?

A small group of activists could volunteer to be our ethical conscience. They prefer story telling and emotional appeal to logic, and being warriors all day long, are very tired of everyone else not understanding. And we can't understand unless we go through the same ideological retraining they have, so anything we do is by default failing to reach them. But since they know better, it should be their job anyway to give us the ethical approval. /s

[challengly]

That's the problem. Those who most want to do that job are the ones who will abuse it the most.

[esalman]

I think what the author is suggesting is that decision making, especially those which can critically affect a person, should not be automated by machines. There must be a mechanism in place which is open to scrutiny (preferably by a qualified human) and accountability.

Take the example of predicting terrorism threats based on facial queues of stress or fear. Machines lack context, which a qualified human would otherwise take into consideration. You can be stressed out because you might be accompanying a child, or fearful that you might miss a flight. If a TSA agent deports someone simply because a machine recommended it, that would be inhumane.

People like to argue that more regulation will adversely affect automation and/or growth/scaling of technologies and businesses. Growth is important but it must not cost us our humanity.

[Verdex]

Personally, I think this type of thing falls into what I like to call the "all problems are management problems" category.

If a TSA agent deports someone simply because the machine recommended it, then that is a problem. But if a TSA agent deports someone simply because they were having a bad day, then that is also a problem.

I guess I don't care what tools those in authority are using (their intuition or a mechanical intuition or a database lookup), but what I care about is whether or not innocent people without the ability to navigate an appeal system are being erroneously penalized.

And ultimately, it's the responsibility of management to make sure that the people/tools they assign are doing a good job. If they fail, then management needs to find new people/tools. If management fails to do that, then all the AI regulation in the world isn't going to do much good.

[esalman]

Well, the purpose of the AI regulation is specifically to hold the "management" accountable when they fail to prevent misuse of the technology. The regulations are to be enacted by the lawmakers. As I see it, the debates like the one we are having right now will determine how the regulation will be shaped eventually.

[Verdex]

Yes, but I'm worried about the people component of my "people/tool" equation. If management sees that costly fines occur when AI is used, then maybe they'll abandon it and just use people instead.

However, if they hire a bunch of power hungry sociopaths who are very good at hiding their malicious oppression and who also bring in donuts every Thursday to stay on their bosses good side, then the situation could easily lead to worse outcomes for the people who have to deal with this system.

If we create a computer system that oppresses 1% of innocent people, then that is a problem. However, I don't consider it a win to ban the computer system and replace it with a human system that oppresses 10% of innocent people. Like, the situation isn't better because humans are oppressing humans instead of a computer doing the oppression.

That's why I was focused on management. I don't care that things are going badly for some specific technology related reason. It's management's job to fix it regardless. If management can't rely on the technology for regulatory reasons, then they might rely on people who do just as bad of a job. And hey, that scenario is even better for management because if they hire a bad actor who gets caught then that person faces the consequences and not them.

[visarga]

Maybe we're going to develop a specialised court system for AI where humans can sue for AI-related injustice. I don't trust companies to self regulate effectively.

[harles]

We haven’t needed such courts for computers, the internet, or other algorithms. I fail to see how special courts are needed for AI. Our current legal system appears perfectly suited for AI - at least no worse than for other tech.

[visarga]

> decision making ... should not be automated by machines. There must be a mechanism in place which is open to scrutiny (preferably by a qualified human) and accountability

You're saying "I don't trust AI, I want human supervision", but this works both ways. Sometimes we don't trust the humans and would prefer a neutral AI. Humans do terrible things to other humans. Who's going to review my appeals? What are their biases? Are they any more trustworthy than a model?

[CivBase]

Agreed. What even counts as "AI that interprets human emotions"? Is a suggestion algorithm designed to maximize clicks "interpreting human emotions"?

[Verdex]

I find this to be an interesting statement. What happens if you're able to evade regulation by making your "AI that interprets human emotions" of really poor quality.

"Your honor, we investigated the source code and discovered that there was technically no AI being utilized. It was a poorly formatted series of if-statements. Honestly, I wouldn't even consider this a program. How it avoids crashing the moment it's run is beyond me."

Like, does AI mean some statistical method is used? That large data sets were used? Are you using some declarative language like prolog?

Ultimately the only definition I can see them coming to will reduce to "person uses computer to do thing I don't like." And somehow I don't think that's going to actually help.

[Hizonner]

Well, I suggest an outright total ban on anything that purports to read people's emotions. Either it doesn't work, in which case it shouldn't be used, or it does work, in which case it shouldn't be used.

[adrianmonk]

Suppose there's a browser plugin that uses sentiment analysis on text fields to try to detect text written in anger. And it doesn't do anything with the information except warn you so you can think twice before posting an angry comment to social media or sending an angry email. Should that be illegal?

Or maybe it more directly tries to sense emotion by using your webcam to look at your facial expression.

[Hizonner]

> Should that be illegal?

That would be acceptable collateral damage, if it couldn't be permitted without opening the door for the creation of systems that used the information against the analyzed people's interests.

I'd be willing to at least consider a carefully crafted exception. The problem being that when you write such an exception, it tends to be awfully easy to introduce loopholes that, in practice, allow using uninformed pseudo-consent, or false consent with no real alternative available, to use information against people.

[PeterisP]

The thing is, if you include something like that (i.e. sentiment analysis as crude, rudimentary way of looking at people's emotions), the genie is out of the bottle - it's a widespread task that's used as a relatively simple homework excercise in undergrad courses, you would need to censor it out of textbooks worldwide, which is a quite big issue to say at least.

I.e. my point is that such a ban would have to be very extensive and invasive, with obvious censorship of small, simple segments of code and whole avenues of basic knowledge. Given some data, you can get a crude emotion detector from facial images or text messages - not state of art but somewhat accurate - with something like ten lines of code, with no previous skill on "emotion analysis", just applying generic ML approaches. I can't imagine how such a ban could be implemented, as so many people would still be able to easily make such systems whenever they wanted to, so the ban wouldn't be effective.

Perhaps you could regulate the application of automated decision making to decisions about people and requiring some review-and-override mechanisms (GDPR has some limited aspects of that), but it's a very different area than just banning knowledge and skills that already exist and are relatively widespread.

[Verdex]

People can read other people's emotions. You can build a mechanical turk program that's effectively the same as having a personal concierge agent with respect to the regulation.

Like, if you ban reading people's emotions you effectively have to also ban any human interaction.

[mitchdoogle]

The problem is making assumptions about why people are experiencing certain emotions, or telling people they are wrong when they say, "I'm actually not angry"

[Verdex]

Yes, but this can happen just as easily with human actors as it can with non-human actors.

I suppose the benefit of a human actor is that you can theoretically fine or jail them if they're found to be malicious or sufficiently incompetent.

However, on the other hand, human actors can explain why they're doing the right thing. Even when they are in fact doing the wrong thing. An AI that is broken incomprehensibly can still be determined to be broken. The human actor causing issues can produce very convincing arguments to avoid termination. Also they can bring in donuts every Thursday to stay on the bottom of the termination list.

[And to be clear. I don't trust the technology at all. I just also don't trust the human system either. A system isn't better because innocent people are oppressed by humans instead of by a computer.]

[Hizonner]

It's still completely reasonable to ban automating it. The impact of having it done in an automated way is completely different from the impact of having it done on a person-to-person basis, for a lot of reasons, starting with scale.

It's also reasonable to ban anything that claims to be better at it than a human.

[Verdex]

I don't think it's completely reasonable to ban it. Although, I would agree that it's completely reasonable to restrict nearly anything.

Although, I do like your comment about scale. If you have a system that's 99% successful, then if you apply it to everyone in the US then you're failing 3 million people. That's a problem.

Of course, your system might be mechanical OR it might just be a group of people each one just "doing their job." From a result oriented point of view, you might end up with a mechanical system that oppresses less people than a people system.

I don't feel good about either one, but I also don't feel good about causing wide scale misery because at least it's people screwing over other people instead of a machine screwing over people.

[Of course it's worth noting that I don't trust the technology at all. It's just that I also don't trust the human solution that it claims it can replace.]

[unishark]

Suppose someone, for example a depressed person, wants to use it to monitor their emotional state and use various automated responses like notifications or distractions to help maintain an emotional state they want?

[causalmodels]

I think this viewpoint is limited. Sometimes reading peoples' emotions can be an extremely good thing.

I am aware of a healthcare company which currently has a model in production which alerts healthcare providers if a person displays suicidal intent. They have several confirmed instances deaths being prevented because of the interventions taken due to alerts from their model. While I don't think ML should be used to manipulate people's emotional states, I think this is a case where having a model that can read people's emotions is a good thing.

[visarga]

The funny thing is that by totally banning "anything that purports to read people's emotions" you also ban research that would help people. For example you need to train models to detect race in order to make sure there are no racial biases in another model you want to deploy.

[adamrezich]

> Regulate how?

make a committee to vote to fund a study to create a subcommittee to vote to fund a study on something from a decade ago with a small, non-representative sample size, reaching predetermined conclusions that justify the need for the committee and all related subcommittees, instead of getting anything done.

y'know, government.

[dj_mc_merlin]

The author is not writing down action items after a meeting. It's the job of politicians (presumably) to draft up policy. Activism groups also push for specific regulation sometimes.

But stating your policy opens up another angle of attack: those who wish to undermine your ideas can attack your implementation rather than the concept.

[pc86]

It's at best sloppy to just say "this needs to be regulated, but I can't be bothered to suggest even one possible regulation that would fix anything I find so objectionable." The author has a doctorate and works at Microsoft Research on AI. If she thinks it should be regulated (which is absolutely a fair position to take) I'd much rather have her be the one suggesting regulations than some 80 year old lawyer who's been in Congress since the civil rights era or some special interest lobbyist.

[dj_mc_merlin]

Her understanding of AI does not translate to an understanding of how a society runs or how laws should be made. If anything, many people overgeneralize their expertise at one task to suggest bad solutions to problems from other domains. Especially engineers and PhDs.

I agree that she's still better than most US politicians. I have some faith in politicians from Western Europe to do half the correct thing, or at least not be swayed by arguments coming from money too much. Not that much faith, but still..

[PeterisP]

If someone wants to make an argument "this needs to be regulated" then they need to assert that the benefits of doing so outweigh the drawbacks. The article does not really do that; it points out a class of potential harm, but it does not try to present an reasonable argument that regulation is likely to succeed in preventing that harm; and the balance of pros vs cons can't even be discussed without at least some general idea about what kind of regulation we're talking about.

[Verdex]

> those who wish to undermine your ideas can attack your implementation rather than the concept.

Yeah, it's frustrating when people do this. But if you want to have real solution at the end of the day, then you'll need to hammer out all of your implementation issues.

If you produce an implementation that is flawed, then people will be able to evade the spirit of your regulation rendering it useless.