[sails]

Snowflake I suppose for the average ML use case. Not for your high-performance ML, but for your average data scientist, maybe?

Edit: I may be wrong[1], would be curious to know what users who've used Spark AND Snowflake would add to the conversation.

[1] https://www.snowflake.com/blog/snowflake-and-spark-part-1-wh...

[marcinzm]

Snowflake hits its limits with complex transformations I feel. Not just due to using SQL. It's "type system" is simpler than Spark's which makes certain operations annoying. There's a lack of UDFs for working with complex types (lists, structs, etc.). Having to write UDFs in Javascript is also not the greatest experience.

[dominotw]

> There's a lack of UDFs for working with complex types (lists, structs, etc.). Having to write UDFs in Javascript is also not the greatest experience.

We load our data into SF in json and do plenty of list/struct manipulation using their inbuilt functions[1]. I guess you might have write a UDF if you are doing something super weird but inbuilt functions should get you pretty far 90% of the time.

https://docs.snowflake.com/en/sql-reference/functions-semist...