| Yes - agree! I actually wrote a blog about this just two days ago: May be of interest to people who: - What to know what DuckDB is and why it's interesting - What's good about it - Why for orgs without huge data, we will hopefully see a lot more of 's3 + duckdb' rather than more complex architectures and services, and hopefully (IMHO) less Spark! https://www.robinlinacre.com/recommend_duckdb/ I think most people in data science or data engineering should at least try it to get a sense of what it can do Really for me, the most important thing is it makes it so much easier to design and test complex ETL because you're not constantly having to run queries against Athena/Spark to check they work - you can do it all locally, in CI, set up tests, etc. |