Hacker News new | ask | show | jobs
by deepGem 1395 days ago
Wow their statement about not participating in benchmarking wars is alarming. In this day and age, when benchmarking tools are so inexpensive and almost everything is very transparent, why not participate.

Or even better engage with a neutral third party such as Jepsen to get on an even playing field and duke it out.
4 comments
datavirtue 1395 days ago
Because their business is providing a solution that IT failed to. Despite the large cost, which the business was already accustomed to from previous IT attempts, pales in comparison to the additional costs of doing it themselves.

It's like the cloud in general, the cost is high but so is the hype. When all that dust settles over the coming years the business will start shopping on price. They will then realize they have been locked in to some extent and will need to start wriggling loose of the lock-in.

link
hodgesrm 1393 days ago
> Wow their statement about not participating in benchmarking wars is alarming.

I found the Snowflake statement pretty reasonable. [0]

Vendor benchmarks are largely propaganda. What actually counts is performance on real-world workloads, starting with your own. Plus good bencharks are costly to do well. If vendors are going to invest in load testing, it's way better to do it as part of the QA process, which directly benefits users. The other thing for vendors to do is to drop DeWitt clauses so others can run benchmarks and share the results. Snowflake announced this in the statement and also changed their acceptable use policy accordingly. [1]

[0] https://www.snowflake.com/blog/industry-benchmarks-and-compe...

[1] https://www.snowflake.com/legal/acceptable-use-policy/

Disclaimer: My company runs a cloud service for ClickHouse that competes against Snowflake.

link
lokar 1395 days ago
Benchmark results rarely predict actual application perf. You need to run your own queries against your own data. Do a real POC.
link
danielmarkbruce 1395 days ago
Because their value prop isn't being #1 on benchmarks. It's about

* being easy to manage * being able to scale up and down compute so you can get good performance without having to keep a bunch of machines running.

link