[ianbicking]

Until reading this article I hadn't realized that emotion detection is banned (edit: but confirmed only in workplaces and educational institutions)

I've had it on my list to try integrating Hume.ai (https://www.hume.ai/) into a prototype educational environment I've been playing with. The entirety of their product is emotion detection, so this must be concerning for them.

My own desire is to experiment with something that is entirely complementary to the learner, not coercive, guided by the learner and not providing any external assessment. In this context I feel some ethical confidence in using a wide array of inputs, including emotional assessment. But obviously I see how this could also be misused, or even how what I am experimenting with could be redirected in small ways to break ethical boundaries.

While Hume is a separate stack dedicated to emotional perception, this technology is also embedded elsewhere. GPT's vision capabilities are pretty capable at interpreting expressions. If LLMs grow audio abilities then they might be even better at emotion perception. I don't think you can really separate audio input from emotional perception, and it's not clear whether those emotional markers are intentional or unintentional cues.

[soco]

Here's a beginner question: what's the big difference between emotion perception and sentiment analysis (which is offered everywhere)? Sentiment goes only plus/minus and emotion produces multidimensional charts?

[Ukv]

Emotion recognition is based on biometric data like facial expression, whereas sentiment analysis would typically be about text:

> The notion of emotion recognition system for the purpose of this regulation should be defined as an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric data.

[ianbicking]

I thought I'd look it up to be sure, finding it in the official PDF [2]: "[Prohibited:] AI systems inferring emotions in workplaces or educational institutions, except for medical or safety reasons."

Elsewhere it specifically calls "emotion recognition" to be of "limited risk" (calling for transparency) and elsewhere kind of implies it to be "high risk" (as being part of the "annex"), though maybe it's just calling out use of emotion recognition in those high risk areas (e.g., credit scores).

But it doesn't seem to actually define "emotion recognition." (Though someone else says it involves biometric data, which seems in line with everything else in the regulation.)

All that said, it seems like under the law you could actually make emotion recognition systems, even for education, it's just that education institutions and workplaces couldn't use them. (Though that's a pretty big blocker for an educational tool!)

[1] https://www.europarl.europa.eu/topics/en/article/20230601STO...

[candiodari]

So there's effectively a blanket exception for the organisations you really don't want to be doing this, ie. Police, the institution itself and government? (incidentally, government is your health insurance company in most of Europe)

I keep coming to the same problem with these regulations. I am much less afraid of Amazon/Google/... figuring out something about me and using it to sell me stuff than I am afraid of the police doing the same, and arresting me or otherwise having a huge negative impact on my life. Knowing the police, they'll probably not even do AI monitoring correctly, and of course, won't be responsible for the damage they cause.

Frankly, that Amazon and Google figure out stuff I might want to buy might actually be a positive. Maybe. Sometimes. If they become better at weeding out scams, that is.

[ianbicking]

While there's a hard block on using emotion recognition in workplaces and educational institutions, the other cases you list are listed under "High Risk" and so they are very much included in the regulation.

There are several exceptions for different kinds of law enforcement, things like eminent danger, or doing biometric filters when searching for someone that fits a description. How much you can squeeze in under those exceptions depends on how bold the police are. Probably a lot, but it's not written that way.