Hacker News new | ask | show | jobs
by danielmarkbruce 1308 days ago
How this is easier than using snowflake/bigquery etc?

The situation where it seems useful is if I'm using a jupyter notebook and I want to use it to pull data in but it's too much to fit into memory, and sqlite is too slow... which seems like a pretty specific situation to be in? And it's not data that is part of some business process that needs updating frequently?

Or I'm just missing the point entirely? I see another post about huge parquet datasets - but how do I get them? Is everyone at my company comfortable with me moving them onto my machine even if I can? Is the dataset so big it won't fit into memory? Can't I just connect my jupyter notebook to snowflake anyway? Then I'm only pulling down the data I want instead of all the data.

I guess maybe the real question is - exactly who is the buyer of this product, how much will they spend on it, and who is the user?