[dwwoelfel]

It has been two weeks since Fastly's VP of Eng called out your ToS and errors in your benchmark in her original tweet thread [1]. I would hope that Cloudflare would have a better response than this that directly addresses Fastly's claims.

Are you going to remove the ToS clause or issue a correction on the blog post?

[1] https://twitter.com/lxt/status/1462896850055352320

[kentonv]

To be clear my comment here is not in any way, shape, or form, Cloudflare's response. I am here representing only myself.

[dcow]

Sure, but you self-identified as someone responsible for the project and then suggested people go violate the ToS by running their own benchmarks. I think the community here calling out the double standards and asking for an update/response is entirely fair. If you didn't want the flak then leave off the "I'm the tech lead on CF workers" intro. Seems to me like the ball is in your court to at least try and make your advice actionable.

[kentonv]

I do not believe I suggested doing anything against the ToS. I think you're misinterpreting the clause. But not being a lawyer, I don't really want to get into that discussion.

If I didn't state upfront that I was the tech lead of Workers, someone would (rightly) call me out for astroturfing.

[justinpombrio]

> I think you're misinterpreting the clause. But not being a lawyer, I don't really want to get into that discussion.

The clause says "Unless otherwise expressly permitted in writing by Cloudflare, you will not and you have no right to: [...] (f) perform or publish any benchmark tests or analyses relating to the Services without Cloudflare’s written consent;"[1]

IANAL, but this seems to very unambiguously prohibit benchmarking Cloudflare's services unless you have written permission. I know you don't want to get into an argument on HN, but could you like... bring it up to someone inside of CloudFlare who would be capable of changing it? You can point to this thread about how this clause is generating negative publicity.

[1] https://www.cloudflare.com/terms/ section 2.2

[eastdakota]

I am a lawyer and also the CEO of Cloudflare. I have no idea why that clause is in our ToS. It was a surprise to me when it was pointed out recently. Not sure when or why it got included. Best guess is it was when some “stress tester” services decided to “benchmark” us by performing DDoS attacks and we thought we needed another justification to shut them off. Has been a loooong time since we worried about such things. Regardless, we decided weeks ago we’re removing the clause during the next ToS refresh. That’s scheduled for the coming weeks. And, in the meantime, have no issue with anyone benchmarking our performance. And seems we should do a more thorough and unimpeachable set of comparisons ourselves. Stay tuned.

[ldoughty]

I appreciate this reply, and hope the initial engineer gets no flak for his personal opinions attempting to defend your company. It's nice to see a tech company with employees defending it and leadership making such public statements as this.

[pbowyer]

At the same time can you change 2.2(a) and remove the "or sign up on behalf of a third party"? My clients are not technical, and when they do manage to sign up they then email me their login details in plain text...

Without this in your T&Cs I could create the account for them in a couple of minutes. And avoid doing a screenshare to walk those who fail through the sign up process.

[unityByFreedom]

Nice. It was a silly jab anyhow, seeing as you need to contact a rep at Fastly to even sign up for their product or get a quote on its cost.

Fastly made it sound like contacting Cloudflare is "impossible", yet here you and one of your top devs are.

[rovr138]

I’m just reading, but I appreciate coming to the comment section of a post here and seeing I’m this interaction, the CEO of a company like CloudFlare posting and, to top it off, posting something like this.

[uyt]

> Regardless, we decided weeks ago we’re removing the clause during the next ToS refresh

I've seen you reply about ToS issues before (specifically over caching of non-html assets): https://news.ycombinator.com/item?id=20791605

You verbally allowed it in that thread but have you considered officially adding that into the next revision of your ToS too?

[wutwutwutwut]

So you haven't read the agreement which you ask all the customers to agree to? Lame.

[Sagaris]

Good move.

As for "stand-up" with Fastly, I believe the whole situation brings only negative consequences on both parties. I always become wary towards any service that posts comparisons with its competitors (or simply with services of similar nature).

Good luck and wise decisions to you.

[ridruejo]

Love these direct founder responses. Makes me trust the service more than any amount of benchmarks

[samhw]

> Unless otherwise expressly permitted in writing by Cloudflare, you will not and you have no right to: [...] (f) perform or publish any benchmark tests or analyses relating to the Services without Cloudflare’s written consent

This - I mean even the conversation prior to this message - surely constitutes "expressly permitted in writing by Cloudflare"?

[saagarjha]

I would not be surprised if the person on the other end of that conversation would take poorly to the existing contributions made to it by people who have publicly identified themselves as Cloudflare employees.

[hn_throwaway_99]

First off, just want to say thanks for your posts, I found they give useful context and I really appreciate them.

I don't want the following to come off as unnecessarily argumentative, but regarding the ToS, I'm not a lawyer either, but my "ability to read English" interpretation of the section on "perform or publish benchmarks..." certainly sounds like it is prohibiting folks from doing their own side-by-side comparisons. Which is, of course, nonsense, because any engineer worth their salt would do their own analysis, even if they didn't publish it.

Just sounds to me like the CloudFlare lawyers got a little too aggressive to the point of absurdity, but I still think it's fair to call out CloudFlare for this.

[ldoughty]

Note that the CEO had replied too this parent thread and said he is removing that language

[unityByFreedom]

Plus it was there to prevent DoS attacks which is understandable.

Even without the ToS language, if you're really going to stress test a service, it's probably a good idea to give them a heads up, lest you get marked a bad actor.

[dmurray]

He still should take some responsibility for it being there in the first place. "Our legal department insists on adding user-hostile clauses everywhere they get the chance" is an OK excuse for a Cloudflare sales rep or engineer, but it's disingenuous coming from the guy who is in a position to tell them to take a friendlier approach by default.

I'm assuming this isn't the only overly restrictive clause in the contract. Maybe it's an anomaly in an otherwise respectful ToS.

[LamaOfRuin]

You suggest performing one's own tests to evaluate performance of the services, which fits the definition of benchmarking.

The ToS explicitly say that performing any benchmark analyses or test relating to the services is not allowed without written permission from CloudFlare.

99% of the people reading and attempting to abide by the CloudFlare ToS are not lawyers, but as a rule contracts mean what they say when they say it clearly and unambiguously as this seems to.

[jefftk]

> If I didn't state upfront that I was the tech lead of Workers, someone would (rightly) call me out for astroturfing.

The way I usually see people handle this, and what I do myself, is to both state your relation to the company and clarify whether you're speaking for the company. Ex: "I'm the tech lead for Cloudflare Workers, but speaking in my personal capacity..." or "I'm the tech lead for Cloudflare Workers (speaking only for myself) ..."

[unityByFreedom]

It's generally understood on HN that when someone says "I work at X" that they are not speaking on behalf of the company.

On top of that, Kenton is a frequent commenter here, and I've never gotten the "air of superiority" vibe from him where such verbosity would be necessary.

[robertlagrant]

This is true, but it's better to just assume people aren't speaking in an official capacity unless they say they are. This would also shave 30% off most Twitter bios.

[dcow]

> If I didn't state upfront that I was the tech lead of Workers, someone would (rightly) call me out for astroturfing.

I think the nuance is that you are presenting yourself as someone with responsibility/authority/control over the subject of these benchmarks. As a comparison, consider wording that skirts taking up that mantle:

Full disclosure, I work for Cloudflare, but ...

Not trying to be argumentative and really don't have any hostile feelings or intent. I understand where you're coming from.. just providing my outside take on how the interaction appears. You're within your right to defend your product. Nobody seems to have a problem with that. But you also decided to throw mud on the pile, metaphorically. You admitted you aren't plugged into the back and forth, that's fine. But this isn't news for Fastly and it's hard to take your side in this discussion when your solution is "go do your own tests" which is exactly what Fastly is would like to do. They clearly call attention to your ToS preventing them from doing that in the piece we're discussing here.

[protomyth]

I think a lot of us have run into this exact clause because of Oracle and then other database vendors. We are well aware of what happens when you violate one of these clauses.

[minusf]

i think it's good there was full disclosure about your employer, but it also set the post up for me as "i am wearing my cloudflare tech lead hat", as nowhere in your post you state you are speaking for yourself and not as an employee... very confusing.

[nickysielicki]

> you self-identified as someone responsible for the project and then suggested people go violate the ToS by running their own benchmarks.

He did not. Profiling how your particular app and use case runs on a given serverless provider is not benchmarking.

[Waterluvian]

FWIW that semantic difference is meaningless to me. A senior member of CF just commented in detail about CF and a competitor.

[kentonv]

Sounds like you have it right.

[ddtaylor]

It seems weird to want to claim authority (eg. that you are the tech lead) but not be willing to also be accountable for your statements.

[saagarjha]

In general, the situation is the exact opposite: companies want to claim authority, but when an employee makes a statement they don’t like they want to have the deniability of “oh they weren’t representing us, if you want our real opinion please talk to our spokesperson”. Corporate PR is a strange mix of wanting engagement but also being incredibly risk-averse, and it’s very different from how people typically communicate.

[bubblethink]

I propose changing these disclaimers about "not representing the company" yada yada, to ICOG, which stands for I am a cog in the machine.

[unityByFreedom]

What statements do you feel he should be held accountable for that he is not? Please quote.

[namdnay]

I’m not sure how much authority being tech lead confers? It’s the lowest possible line management position. I wouldn’t expect a tech lead to have any influence whatsoever on legal, contractual or communication issues

[NicoJuicy]

If you know a little about how corporations work, than a tech lead is not responsible for a ToS, probably doesn't know anything about it and that's fine. Since it's not expected either.

It's the legal department... And the CEO already mentioned that they are removing it and he gave a valid response/reason.

It seems that they will actually benchmark Fastly in detail now ( could be after another improvement week), which probably isn't what Fastly wanted.

Something definitely seems to be happening if you read their response and i'm awaiting it with actual stats!

https://news.ycombinator.com/item?id=29468771

@dwwoelfel that's what you wanted? :)

[dwwoelfel]

My experience working at tech companies is that the tech lead, or anybody at the company, can post in an internal message board or slack to ask "what's up with this weird clause in our ToS" and expect an explanation.

It's nice that eastdakota responded here, but he had two weeks since the original tweet thread from Fastly's VP of Eng calling out the problems with their benchmarking. They didn't respond or retract the blog post in those two weeks.

As a cloudflare shareholder (and a fastly shareholder), I want Cloudflare to act ethically and either retract the blog post or issue a correction.

[NicoJuicy]

And what i read from his comments is that a follow-up post will come.

You are insinuating bad will/faith and that's not the impression i observed.

[dwwoelfel]

Cloudflare's blog post still says, in bold, "Cloudflare Workers is 196% faster than Fastly’s Compute@Edge based on the time to first byte from the tests we ran on 50 nodes using Catchpoint’s data from across the world".

It is unethical to leave that up after Fastly pointed out core issues with the benchmarking, like using a free tier that was rate-limited.

[kentonv]

Cloudflare's test compared the free tier of both services. The post was explicit about this. Workers free tier has limits too, and we would certainly have preferred to use the paid version of Workers in our test, but as the paid version of C@E is only available with an enterprise contract, the only fair test we could run was between free tiers.

Incidentally, this means Fastly's blog post is currently displaying test results that compare the enterprise version of Compute@Edge against the free version of Workers. Granted, our bad for the ToS clause, but still.

Despite the strong language in their post, Fastly has not actually demonstrated that anything was intentionally biased or unfair in Cloudflare's test. They've only laid out their opinions as to what would make a more representative benchmark. That's a debate you can have about any benchmark, but that doesn't somehow make the original benchmark "unethical".

[dwwoelfel]

It's not the benchmark that's unethical. The unethical part is leaving up the original claim without adding a correction or a note that addresses the problems Fastly found with the benchmark.

[staticassertion]

I wouldn't expect everyone at Cloudflare to be intimately following every competitor's blog posts tbh

[dwwoelfel]

But I'd bet kentonv was following this one, since he said "until Fatly complained about it" and not "until I read the blog post".