[dwwoelfel]

Cloudflare's blog post still says, in bold, "Cloudflare Workers is 196% faster than Fastly’s Compute@Edge based on the time to first byte from the tests we ran on 50 nodes using Catchpoint’s data from across the world".

It is unethical to leave that up after Fastly pointed out core issues with the benchmarking, like using a free tier that was rate-limited.

[kentonv]

Cloudflare's test compared the free tier of both services. The post was explicit about this. Workers free tier has limits too, and we would certainly have preferred to use the paid version of Workers in our test, but as the paid version of C@E is only available with an enterprise contract, the only fair test we could run was between free tiers.

Incidentally, this means Fastly's blog post is currently displaying test results that compare the enterprise version of Compute@Edge against the free version of Workers. Granted, our bad for the ToS clause, but still.

Despite the strong language in their post, Fastly has not actually demonstrated that anything was intentionally biased or unfair in Cloudflare's test. They've only laid out their opinions as to what would make a more representative benchmark. That's a debate you can have about any benchmark, but that doesn't somehow make the original benchmark "unethical".

[dwwoelfel]

It's not the benchmark that's unethical. The unethical part is leaving up the original claim without adding a correction or a note that addresses the problems Fastly found with the benchmark.

[NicoJuicy]

But it still compares both free tiers as he mentioned. It seems dubious that you demand to compare Fastly's paying tier with Cloudflare's free one ¯\_(ツ)_/¯

Cloudflare's free tier does not optimize for speed/performance, but for available ( = unused ) datacenter capacity based on location. Which makes it less fast than their paying tier.

Additionally, there will be an update soon as mentioned before, based on past comments.

There are other things that you ignore/are unaware of that are not even mentioned in Fastly's post. Eg. That cloudflare also optimizes their network for routing to denser cities instead of rural areas. That metric is not even mentioned by Fastly...

Note: i don't know any inner workings of them as I don't work there. It's based on what I remember from their blog about their SDN and performance weeks. I suppose it's applicable to this scenario, if i got the details right.