Hacker News new | ask | show | jobs
by Mortiffer 937 days ago
Based on recent experience with large deployments at S&P500 types, IMHO the value snowflake providers to most companies can easily be deployed in house with either old school Hadoop,Spark, Hive or old school Impala, Hbase or more recently Trino.

The clients I had where we installed snowflake every is super worried about any additional copy operation because everyone is keenly aware of the unlimited cost. This makes getting shit done harder.

Clients that have a decently sized k8s cluster running Trino or old school Spark/Hadoop cluster on fat VM's you know what you got you know what your paying for you make estimate of how much ram / cores you need for certain workloads and once they are purchased your engineers get really good at squeezing as much work out of the given resources as possible. And no constant complaining in meetings about what potential extra cost this additional computation may have.

Also if other people working in other parts of your org don't run snowflake themselves you gotta pay for their snowflake usage on your bill or you pay for copying data back out to S3/ADLS/sFTP so that other departments can get to the results of your computations. And snowflake really doesn't like it when you do this, they even gave exporting data a new name, they call it "unloading" making you feel like your undoing something which you probably should not undo.... On that note, Snowflakes data export options are significantly underdeveloped in contrast to Databricks, Cloudera and also the original open versions Spark, Trino and Impala.

</rant> :)