[marginalia_nu]

On a theoretical level, a service like Cloudflare is the most terrifying entity on the Internet I'm aware of. They've accumulated an insane degree of insight into the traffic flow of the web (since their entire service is essentially acting as a HTTPS middle man), and their business is offering protection against bot spam that could ruin most websites. Even if they aren't operating the bots themselves, they're essentially displacing the bot problem to the unprotected websites. Like the overall shape of this operation is something the cosa nostra could have cooked up in the 1970s.

However, being on both sides of this, both operating a bot for my search engine, and operating a web service that is aggressively targeted by bots. They're not actually bad to deal with.

The big unanswered question is how they'll manage to stay good given the obvious incentive of abusing this setup. Maybe this CEO has a moral backbone, but will the next, and when they're acquired by the Meta-Amazon-Alphabet group in 15 years, will they still stick to these principles?

[mst]

Internet security has, in my experience, always been about "being just hard enough a target the bad actors decide to go torment somebody else."

It was true twenty years ago too, the only difference I can see between then and now is that you can outsource that task for a (relatively) small amount of money if you want to.

Then again, the last time I dealt with a site under DDoS, something in their stack was leaking the underlying IP (never did figure out what) but it turned out that "finding a provider who'd sell them a decent sized server and charge them for the bandwidth" was perfectly economical for their use case because their haters' firepower was insufficient compared to their revenue.

(I'd love to be less vague here but I'm sure readers can see the obvious professional ethics issues with doing so)

[WirelessGigabit]

I'm surprised you're handing incoming requests from everybody. We only process the CloudFlare ones and drop the rest.

[lillecarl]

You can fill the pipes to the server(s) you're targeting, it doesn't have to be application layer.

[yencabulator]

These days, Cloudflare lets you serve your origin via a tunnel from a host that doesn't even have a public IP.

And if you run that in a cloud, the NAT isn't your problem -> your attacker will have to DoS that cloud as a whole.

[mst]

That's an extremely smart approach that I sincerely doubt the site operators would have been capable of dealing with.

Part of the art of consultancy is, sometimes to my great annoyance, optimising for "within the customer's budget" and "within the customer's capacity to maintain it after I'm no longer involved" over "best possible solution."

Plus in this particular case I was working pro bono because (a) I quite liked the site in question continuing to exist (b) a Shadowcat alumnus asked me nicely (c) I take great pleasure in ruining a griefer's entire week. So lightest possible touch was strongly indicated.

The end result was not remotely clever, but it's been in production for a while now and has not to my knowledge caused financial or uptime issues, so I'm going to call it a win even if the inelegance of -how- I won continues to irritate me ;)

[mst]

Right, "finding a provider whose reaction to an aggravating quantity of incoming packets was charging money rather than throttling the connection" was basically the load bearing part of the solution here.

Fortunately, while said quantity was indeed aggravating, it was low enough that the cost was financially and logistically less than trying to do something more elegant.

Sometimes brute force and ignorance is, in fact, the right answer, and I don't have to -like- that being true for it to be true.

[quanticle]

>The big unanswered question is how they'll manage to stay good given the obvious incentive of abusing this setup.

Why do you think they're still "good"? CloudFlare has chosen to abandon sites that held free speech (abhorrent speech, but still free speech) while still protecting forums upon which credit cards and methamphetamine were listed for sale on the front page.

To me, that's not a sign of a "good" actor.

[waboremo]

Free speech doesn't exist within the context of a privately held website.

[marcellus23]

Free speech is an ideal, not just an amendment.

[riffraff]

but a private entity (person or corp) does not have any obligation to protect ideas they find abhorrent to be considered on the side of "good".

[iceburgcrm]

That's the point. They find free speech abhorrent but consider selling dangerous drugs to be acceptable.

So many people find them abhorrent because they represent different values.

Legally no one has any obligations here.

[quanticle]

Agreed, but it's a strange value system that says, "Dealing meth and stolen credit cards is okay, but having a web forum that makes fun of people is not."

[TobyTheDog123]

I always figured that the main thing Cloudflare protected against was DDoS attacks, not bots (DDoS may be caused by bots, but with significantly different outcomes -- a single bot in and of itself won't take down a website)

RE bots: TikTok has incredible bot protection that comes from engineering (webmssdk) instead of network-based filtering. I'm not even sure if they use Cloudflare.

[jart]

Cloudflare doesn't even really protect against DDOS. Sometimes taking your website off Cloudflare is the only way to stop a DDOS attack. That's because you can't stop something like a level 4 ddos attack by blocking the IPs in raw prerouting iptables, because if you did that then you'd be blocking Cloudflare's IPs. The only option Cloudflare really provides you is pressing a panic button that forces everyone who visits your site to view a captcha, when it's really so trivial to just run the iptables commands using a token bucket algorithm. I know because I run a website on a 2 vCPU VM that gets DDOS'd all the time. I've had to block over nine thousand malicious malicious IPs so far. I tried using Cloudflare in the past for their protection services, but it made me (1) defenseless against bad visitors and (2) made good visitors angry at me for the captchas.

[solardev]

How did the attackers get your origin ip to begin with? I thought cloudflare was supposed to shield it at the DNS level, and in theory your origin should be dropping all connections not coming from an authenticated Cloudflare proxy?

[jart]

They weren't able to talk to my origin IP, because when I was using Cloudflare, I blocked at the firewall all IPs that weren't Cloudflare. The problem is that they would DDOS my server through Cloudflare. And because the traffic was being proxied, I couldn't block the attackers without blocking Cloudflare. Unless of course I wanted to fill out a form on their website 9,000 times. It's an awesome website by the way. I love their workers and r2 products. But Cloudflare honestly isn't that good at DDOS protection. These attacks were so bad that Cloudflare would start showing NGINX error pages before my web app even went down. Cloudflare should be paying me to protect them, rather than the other way around.

[prdonahue]

Do you have a support ticket # you can email me w/details (pat at cloudflare)?

We take every reported false negative as an opportunity to improve our DDoS mitigations, and these reports are very helpful.

As of a few weeks ago, you can now report FNs/FPs for Bot Mitigation directly in the dashboard, and we'll be expanding this pattern for use with DDoS Mitigation as well.

[solardev]

They do both. Ddos mitigation happens at the network level, while bot protection uses a combination of whitelists, blacklists, behavioral heuristics like mouse movements, login state, and captchas.

[AtNightWeCode]

ALL big tech companies have the same setup. There is nothing unique with Cloudflare. People are just talking about Cloudflare cause it is accessible for free and they sell it as a service.

[ZoF]

He has shown time and again that his backbone's strength depends on how loud the public noise is. Kiwifarms most recently. You can dislike them(kiwifarms etc) and there is a case for them to be taken offline imo, but it is the governments job.

Exactly what you do _not_ want protecting the neutral internet. They've done better being neutral than some might have, but that's in reality more insidious because clearly there are points they will bend on and those points will change over time and almost certainly continue to erode.