Hacker News new | ask | show | jobs
by kentonv 1658 days ago
Cloudflare's test compared the free tier of both services. The post was explicit about this. Workers free tier has limits too, and we would certainly have preferred to use the paid version of Workers in our test, but as the paid version of C@E is only available with an enterprise contract, the only fair test we could run was between free tiers.

Incidentally, this means Fastly's blog post is currently displaying test results that compare the enterprise version of Compute@Edge against the free version of Workers. Granted, our bad for the ToS clause, but still.

Despite the strong language in their post, Fastly has not actually demonstrated that anything was intentionally biased or unfair in Cloudflare's test. They've only laid out their opinions as to what would make a more representative benchmark. That's a debate you can have about any benchmark, but that doesn't somehow make the original benchmark "unethical".
1 comments
dwwoelfel 1658 days ago
It's not the benchmark that's unethical. The unethical part is leaving up the original claim without adding a correction or a note that addresses the problems Fastly found with the benchmark.
link
NicoJuicy 1658 days ago
But it still compares both free tiers as he mentioned. It seems dubious that you demand to compare Fastly's paying tier with Cloudflare's free one ¯\_(ツ)_/¯

Cloudflare's free tier does not optimize for speed/performance, but for available ( = unused ) datacenter capacity based on location. Which makes it less fast than their paying tier.

Additionally, there will be an update soon as mentioned before, based on past comments.

There are other things that you ignore/are unaware of that are not even mentioned in Fastly's post. Eg. That cloudflare also optimizes their network for routing to denser cities instead of rural areas. That metric is not even mentioned by Fastly...

Note: i don't know any inner workings of them as I don't work there. It's based on what I remember from their blog about their SDN and performance weeks. I suppose it's applicable to this scenario, if i got the details right.

link