[narrator]

The API call that analyzes what's going on in an image will get sent to China and analyzed there by no-name state owned mega surveillance corp and the result sent back like "35/m/$50k income/55% criminality likelihood/60% likely homosexual/Woman standing next to him is 35% likely to be mistress/subject is currently engaged in 30% flirting behavior 20% shopping for clothes/Target with ad for greek vacation/" and they'll all say it's just a model. Pinky swear there's no personal data that was used to look that up in China.

[henrikschroder]

It doesn't matter if that lookup is done in China or in the EU or elsewhere. To be compliant a company has to be able to list all third parties that get access to personal data it collects, and what they in turn do with it and further third parties that get access to it.

If a company uses a Chinese mega surveillance corp API, they still have to disclose it, and they can't just hide behind a "the computer says no" response if they use the results of that API call to make a business decision. The GDPR gives the data subject the right to know why and how the business decision was made, and gives the subject the right to appeal.

[narrator]

So there was a 5 petabyte learning set we trained a 20 layer deep neural net and a value came out the other end. Here's all the math we did to figure that out (several gigs of float computations). We don't even know what it's doing exactly. What does GDPR say about that? Is feeding data into a deep neural nets illegal in Europe now because they lack explainability?

If you took someone's picture and ran it through neural style, would that be illegal because you couldn't tell them exactly why it painted their nose blue while imitating Leonardo Da Vinci's artistic style? Is Google auto identification of objects in personal images illegal now because they can't explain how a deep neural net works and classified their friend as something non-human by accident? This has actually happened.

[henrikschroder]

> So there was a 5 petabyte learning set we trained a 20 layer deep neural net and a value came out the other end. Here's all the math we did to figure that out (several gigs of float computations). We don't even know what it's doing exactly. What does GDPR say about that?

That depends completely on what you are using the value for. Recommending five funny articles - GDPR does not apply. Denying an insurance claim - GDPR most certainly applies, and you have to be able to explain what factors went into the decision. You can't have unaccountable oracle boxes.

Note that a perfectly valid answer could be something like "We've analyzed your posts on social media and we've categorized you as having severe anger issues, which is why we're denying this auto collision insurance claim, because you are most likely at fault given situations like this". The GDPR says you have to be able to explain your decision like that. It doesn't forbid you from making these decisions.

> If you took someone's picture and ran it through neural style, would that be illegal because you couldn't tell them exactly why it painted their nose blue

This is not a business decision, don't be ridiculous.

[narrator]

The "posts" -> "you have severe anger issues" decision sounds like a black box to me with the way NLP models have been going.

The Chinese Corp API seems like the same thing. "We looked at you and decided you look like someone who wants to go on a Greek vacation."