Hacker News new | ask | show | jobs
by Nextgrid 1482 days ago
If you want to benchmark for internal reasons you don't publish the results and nobody knows. If you want to make a service to the community, run your benchmarks, download Tor and publish the results anonymously. I don't see what the big deal is?

Is this only limited to marketing claims where you post it on your company's website?

4 comments

The only way I can openly talk about a service's performance is by doing it illegally and you don't find that weird?
How likely is it that one takes an anonymous benchmark published by a noname researcher seriously?
Ah the same level as if it were from an acclaimed researcher: not only did they do the work, they also risked breaking the law to disclose it.

This isn't line the traditional case of no name vs trusted name: due to the law you _must_ br anonymous to post this, so anonymity isn't a red flag, it's the standard.

Extremely.

Why would you assume people generally do background/credential checks on researchers?

Sure its nice to have it on phoronix or something, but its by no means a deal-breaker if it isn't.

Aren't benchmarks supposed to be more or less reproducible? So just publish the results and the data?
100%

Have you met programmers?

Its mainly related to MSSQL and Horracle. Horracle will just use their legal team (which is bigger than their engineers and developers) to bludgeon you over benchmarks.
"This just in Oracle legal team takes down entire Tor Network"

Jokes aside, I'm surprised they're so touchy about these things. They can make plenty of money without it, they can also save plenty of money with less lawyers.

If rationality was always used we wouldn't have had Putin making a gigantic, catastrophic geopolitical mistake. A little common sense goes an awful long way, but only if you choose to use it.

MSSQL is destroying its own market by its pricing (and the licensing thicket, jesus christ, I've been looking at them for 2 days now and... shudder)

Used to just use PostgresSQL on Azure just fine. I am really surprised people use MSSQL or Oracle today honestly. You really don't need to tie yourself down to a proprietary DB anymore.
> I don't see what the big deal is?

The big deal is that it's slanderous.

Oracle isn't making an attestation about performance that the benchmark addresses, the benchmark aims to make statements about Oracle for some other reason, and that's important: Rights to free-speech generally end when they cause harm, and bearing full costs in the defense of false benchmarks is certainly harm.

Look at it this way: The clause aims to prevent purchases (entering into agreement with Oracle) under false pretenses. Oracle sells software to solve business problems, not you-need-a-paper-that-makes-Oracle-look-bad problems, and I think vendors are wise to protect themselves from that.

On the other hand, if you actually bought Oracle to solve a problem, and it didn't do that, you're still free to make those benchmarks and sue the shit out of Oracle with them, and this agreement can't by itself prevent the benchmarks from reaching the public record at that point.

> Is this only limited to marketing claims where you post it on your company's website?

If your company makes X and your company website contains a benchmark saying Oracle is slower than X, you're not just making a statement that you observed Oracle was slower than X, you're also making an attestation that the benchmark is a fair representation of both Oracle and X. And judge and jury are going to be wondering if it's as fair as you say, or if it's unfair as Oracle says.

Now, if you're a university and you don't make X, you might be able to argue that even if it's unfair, it was done in good-faith, and judge and jury may believe that, but Oracle will ask, if you truly believed X was fair, why didn't you get our feedback before publishing? and you'd better have a good answer to that.

On the other hand, if you choose to be anonymous, you may be able to avoid the judge and jury, but the community has to wonder who you are, whether you are motivated by a relationship to a company or product that competes with Oracle, or an impatient researcher who can't meet the standard of professional publishing. The community will wonder, but they have lots of other things to wonder about too, so they probably will not wonder for very long. So what's the point? Techdudes already know what they think of Oracle, and nobody who writes code talking or Oracle thinks that Oracle was chosen for its benchmarks, so who is this anonymous benchmark for?

And here we can see Oracle lawyers in their natural habitat, spewing bullshit rhetoric to protect them against normal usage of their proprietary software.
>The big deal is that it's slanderous.

Then you don't need to forbid benchmarking in the terms, if someone posts a slanderous/libelous benchmark then sue them for that.