[tyingq]

There was a recent example with Google Drive where it explicitly disabled any way to appeal. I was able to reproduce the issue where it was flagging files that consisted of a single byte, sometimes followed by \r\n or \n.

Here's the HN story: https://news.ycombinator.com/item?id=30060405

Screenshots of trying to "appeal" (Request a review) from when I recreated the issue show pretty clearly there is no human involved: https://imgur.com/a/5YHQtLi

This wasn't an account ban, so I don't know how well it fits the GDPR language. Though I'd be surprised if this was somehow the only "fully automated account action" FAANG type companies are doing.

[Someone]

I don’t see how you get from Google’s statement “Was taken down for legal reasons and cannot be appealed“ to “no human was involved”.

[kelnos]

I think the point is they, based on the file content, no human could have been involved in the decision. If there was a human involved, the files never would have been flagged.

[Someone]

The discussion isn’t about the initial flagging, it’s about the review.

That might be as simple as checking for the existence of legal documents claiming copyright infringement, or as reading a web page stating “we already removed X other copies of this file”.

Neither is a fail-safe way of doing such a review, but doing a thorough review might be expensive even for Google. Does anybody know how many such reviews they do each day?

It might also be a bug on their tooling to assist human reviewers.

[kelnos]

Right, but I believe for those cases, the (automated) email also said that the ability to appeal was not available. So there is no human review, because there's no review at all.

Regardless -- and I know this is a "how the world should be, not how it is" type thing -- I really think the initial decision should not be allowed to be made by an algorithm. At the very most, an algorithm should be allowed to flag something for human review, but no action is taken until the human has a chance to review it and decide if the flag is warranted or not.

[SAI_Peregrinus]

You're assuming the human both has agency, and gives a damn. It's more likely the human just rubber-stamps all bans, to get their KPI of number of appeals processed per day up!

[saghm]

If the human doesn't have agency, then it's not really a "human review", is it?

[rat9988]

It still is.

[foxfluff]

Spirit of the law is a concept I would encourage anyone to think about when arguing about these things. I believe most people, and courts in particular, would not agree that a human rubber-stamping automated decision is in line with the spirit of the law. Clinging onto a technicality isn't going to go well.

I'd also like to point out that these laws don't just come out of nowhere in a vacuum, to be interpreted without any further context. In EU we have recitals and guidelines to give context and support the interpretation of regulations.

If you're interested, do read Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (wp251rev.01).

https://ec.europa.eu/newsroom/article29/items/612053/en

Here's what it says about human intervention: "Any review must be carried out by someone who has the appropriate authority and capability to change the decision. The reviewer should undertake a thorough assessment of all the relevant data, including any additional information provided by the data subject."

[philipov]

And you've just demonstrated why programmatic enforcement of contracts doesn't work. Courts are actually able to see through that semantic nonsense, because humans are able to discern intent.

[throwaway22032]

Here's a human review process for you:

1) Check whether the output of the machine learning model outputs Yes or No.

2) If yes, ban.

3) If no, no ban.

This is not a human review process. The review process is algorithmic, the human is only involved to relay the result.

[saghm]

I think the chances that a judge would accept that argument is potentially a lot lower than you might expect.

[bsedlm]

it may still be human, but if it's just rubber stamping is it really a review?

[foxfluff]

They are required to have agency and give a damn. Of course, it is hard to (dis)prove that they actually do.

[mdoms]

That's an automated process then.

[tyingq]

Because once a human did get involved, via noise from here and twitter, Google admitted that there was a problem.

Also, just the absurdity that a human would review a file containing only "1" and decide the decision to flag it was correct.

https://twitter.com/googledrive/status/1486038872928792576

[godelski]

What's interesting is there's another user there that followed up 2 days after the tweet noting that other numbers weren't fixed. But this is ignored.

Problems shouldn't get fixed just because they got enough likes and reshares on Twitter.

[jeffbee]

Are you suggesting that Europe has established a fundamental human right to have Google provide free static hosting services?

[gpm]

No, they're suggesting that given that Google has chosen to provide free static hosting, Europe has decided they can't moderate it with purely automated systems with no appeals process.

This is like running a restaurant in the US, and not being able to discriminate by race. You're not required to run a restaurant, and certainly aren't required to run one that gives away free food, but if you are certain obligations come attached.

I'd also argue that your use of the phrase "fundamental human right" is misleading. Europe can and does require you do things for reasons other than respecting fundamental human rights. So does pretty much every other law making authority.

[tgsovlerkhgsel]

The EU has established that "The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her." (https://gdpr-info.eu/art-22-gdpr/)

[Retric]

Arguably the opposite where the EU may have in effect outlawed may free services be requiring human review of many activities.

[foxfluff]

I think reading the part where they say "I don't know how well it fits the GDPR language" would answer your question.