[kstrauser]

Honestly, the presence of that clause screams to me "this app sucks and we'll sue you if you tell anyone how badly". That may not be the case whatsoever, but my first assumption is that they're trying to hide terrible performance.

[jandrewrogers]

While I would prefer if this clause was not a thing, I also understand why it exists even for great products.

It is surprisingly difficult to reproduce many workload benchmarks and quite easy to engineer a benchmark that misrepresents real-world database performance. There are tools that exist to generate optimally pathological workloads that target specific database implementations, while looking completely reasonable and innocuous. It doesn't even need to be a bad faith benchmark by a competitor, there is a high probability that the person configuring the environment does not know how to do it correctly and/or optimally.

The DeWitt Clause is a defense against the unfortunate pervasiveness of incompetent and/or bad faith benchmarking. Companies have a well-founded reason to not trust third parties to do a good job of representing the performance of their product.

[kstrauser]

I understand your point, and that’s a reasonable argument. I do disagree with it, though.

Imagine a hypothetical FooDB by Bar, Inc. If Bar never put that clause in the FooDB license, then I think you’re absolutely right. People would come up with some awful-looking benchmarks that made it look bad. However, what a golden opportunity for Bar! They could step up with some free or steeply discounted consulting to help the benchmarker fix the problem and publish new, good results. They wouldn’t have to do that too many times for word to get around on sites like this: FooDB is nice and fast when you tune it correctly! That would come along with some enormous goodwill, and also the assumption that if your FooDB installation is performing poorly, then it must be your fault because all the benchmarks say it’s really fast for everyone else.

I’m not going to tell Bar what their business model should be. I have my thoughts on it, but it’s their business to run as they see fit. But if I see Bar being open and helpful with a freely-accessible tech blog telling you how to make FooDB stand up and dance, I’ll tend to believe that it’s probably an interesting product to look at. If they guard those secrets behind a wall of lawyers and sue people who speak ill of FooDB, I’ll tend to believe they’ve got something to hide. Either one of those beliefs might be completely wrong, but that’s still how I’m likely to perceive it.

[hyperhopper]

I don't think this is a reasonable argument: bad faith stats and data manipulation exists in literally everything.

We should all be free to express our thoughts and backing data, and participate freely in the marketplace of ideas.

No corporation should be able to put gag orders on people especially when they are biased and have good reasons to want to control the discourse

[tempodox]

If a pathological workload can look completely reasonable and innocuous, then what's the difference between a bad-faith benchmark and a user making an honest mistake?

[ChikkaChiChi]

Reproducible benchmarking can also inform the person/team that their configurations are suboptimal. How are you supposed to know how good your "best" is if you aren't allowed to talk about shakedowns?