[gonehome]

IIRC there was a follow up which was arguably worse.

Google implied that the account was suspended because of real CSAM concerns unrelated to the photos sent to the doctor/police officer (other photos in their account? they wouldn't go into details about their decision) and that the officer closing the case doesn't mean their judgement here is wrong.

If that's the case I can understand why they're obstinate about their decision (which otherwise would seem like a dumb mistake they should just reverse), but the problem is none of this happens in a place where users have any ability to get reinstated or have any sort of control over their digital life - there's no real path any individual has out of this even after going to the press. There's no due process, no way to defend yourself, no way to get them to show how they made their decision. As I understand it you have no rights beyond the ToS.

The user also losing all related account access (two factor, email, etc.) is particularly bad. This is also categorically different from Apple's approach which compared hashes of CSAM with what's in the NCMEC database which would not have caused this mistake (here Google is using computer vision to discover and flag novel images).

[skissane]

> There's no due process, no way to defend yourself, no way to get them to show how they made their decision. As I understand it you have no rights beyond the ToS.

This is where legislation is required. Mass-scale social media/cloud providers/etc are effectively public utilities, and you should be able to challenge their decisions in court - the current situation is as bad as if the electricity company could disconnect you for non-specific reasons and you had no reasonable prospects of a successful legal challenge to their decision.

(Such regulation should be limited to providers above a certain size-maybe a cutoff like 10 million MAU or 500 million annual revenue-so small players aren’t burdened by it.)

[fncivivue7]

That's spin. The real reason is discussed near the end of the article.

They don't want to get into the business of deciding what is and isn't sexual imagery. Instead, it's easier to just ban people and forget about it.

It's the same type of behaviour and attitude that lead to damoore being fired. No room for ambiguity or nuance at Google. Everything can be decided by an algorithm.

[gonehome]

If the reason stated at the end of the article is the reason I can also understand their decision.

If my SO was surreptitiously taking nude pictures (videos?) of me and our kid while we were sleeping (and uploading them to the cloud!), I’d be pissed (and this is the most benign interpretation).

That said, there should still be due process.

Ultimately, I think it’s an issue with the local max of computing we’re trapped in. We need tools that can free us from dependence on a handful of centralized companies that have this kind of discretionary power over our lives.

[fncivivue7]

I dunno, cloud or no cloud there is an expectation of privacy. What they've done amounts to a warrantless fishing expedition and it should concern everyone.

The problem is, the alternative is this stuff runs rampant on their service, not something I'd want either.

So I understand their position, but their approach here is lacking.

[dmix]

Zero transparency and zero due process is a massively underrated feature of big tech censorship/policing.

It's one thing to debate the benefits that society gets for removing/flagging content, but the costs are immediately 10x higher when the false positives have zero recourse.

Even 'legitimate' cases often can't speak to a human to find out why or how they were flagged (unless they have influence). The public can't find out what motives/reasoning or who was behind certain content being removed. So it feeds into conspiracies and builds resentment.

[Buttons840]

Morally speaking, once a company becomes big enough they should owe a minimal standard of customer service to society.

I'd hope to see this covered under anti-trust laws (not a lawyer though, don't know the laws or their applicability). There's only one Google, and Google had quite the monopoly on being Google. They're big enough to be either regulated or broken up.

[rmbyrro]

I wonder what images they used to train an AI to recognize a child penis.

[trasz2]

There is no reason for Google to not be lying about it.

[thecoppinger]

I faced a somewhat similar predicament earlier this year.

I still have no idea how, but my 2FA enabled Facebook account (with a unique and secure password) was compromised while I was sleeping. Shortly after, the attacker started using my Business Manager account to run ads for fake products on their scam stores.

Here's the catch: my personal Facebook account was permanently banned right away, but my Business Manager account wasn't.

How? According to other folks who had the same done to them around that time, the attackers would upload CSAM content on your timeline so that you get immediately banned/locked out.

Well, that means I could no longer retrieve/change my business manager account, which gave the attacker free reign to run ads for about a month. To some degree this means that Facebooks CSAM system gives the attackers a way to compromise Business Manager accounts more efficiently.

I submitted a ban appeal, but didn't hear back. I read online that if you have an Oculus, reaching out via their support is the only real option, so I did just that.

I wrote down a detailed account of the timeline of events, along with screenshots etc., and sent it to an Oculus support agent. In fact, they thanked me during the interaction or providing 'the most detailed' report they'd seen.

The evidence was pretty clear: at 5am or so, someone had logged in to my account via a foreign IP, change my email to a Chinese address and added a hardware 2FA key. The ads they were running to scam stores were often in Chinese, too. Not exactly a difficult case to crack.

They assured me I'd hear back within 7 days, but a month or so later I received an automated email from Facebook stating that the time for my appeal had expired, so the account would stay permanently banned.

That was mildly infuriating, given I never heard back from anyone.

What did losing my Facebook account mean to me?

As much as I'd been considering moving off social media, it briefly ruined my life.

* I'd had my account since I was 13 years old in 2008. I had a few thousand connections on there, many fleeting and superficial, but at least a few hundred with folks around the world that I care about and have no way of reaching now.

* 90% of communication here in NZ transpires via Facebook Messenger, so I was immediately cut off from my community and friends. What's worse, many have since told me that they were worried I'd blocked them.

* My income from the time came from selling trading cards in FB groups while I was closing an investment round. I lost the ability to do so, and had to move out of my apartment to live outside of the city with my in-laws.

* My father passed away a few years ago, and I had countless photos of him on my account, as well as our message history. This honestly hurt more than anything else.

All in all, this experience has left me a deep scar. I guess I needed to learn a lesson around not relying on one platform so heavily, and to some extent not backing things up such as the photos, but I really wish Facebook could have just done the reasonable thing and let me back in.

Finally, I have no idea if I was reported to the police/LE in any capacity regarding whatever was posted on my account to have me banned. Am I on some kind of list now?

A boring, technocratic dystopia.

edit: on the off chance anyone from Meta reads this and thinks they can help, I would be over the moon to get even a chance of having my account restored. I was told to speak to an Australian law firm who charge $3,500 to hound Facebook to get accounts restored in situations like mine, but unfortunately that's just not within my means.