[kstrauser]

I understand your point, and that’s a reasonable argument. I do disagree with it, though.

Imagine a hypothetical FooDB by Bar, Inc. If Bar never put that clause in the FooDB license, then I think you’re absolutely right. People would come up with some awful-looking benchmarks that made it look bad. However, what a golden opportunity for Bar! They could step up with some free or steeply discounted consulting to help the benchmarker fix the problem and publish new, good results. They wouldn’t have to do that too many times for word to get around on sites like this: FooDB is nice and fast when you tune it correctly! That would come along with some enormous goodwill, and also the assumption that if your FooDB installation is performing poorly, then it must be your fault because all the benchmarks say it’s really fast for everyone else.

I’m not going to tell Bar what their business model should be. I have my thoughts on it, but it’s their business to run as they see fit. But if I see Bar being open and helpful with a freely-accessible tech blog telling you how to make FooDB stand up and dance, I’ll tend to believe that it’s probably an interesting product to look at. If they guard those secrets behind a wall of lawyers and sue people who speak ill of FooDB, I’ll tend to believe they’ve got something to hide. Either one of those beliefs might be completely wrong, but that’s still how I’m likely to perceive it.