Hacker News new | ask | show | jobs
by sspiff 1008 days ago
Perhaps slightly off topic, but I'm not familiar with DuckDB. I don't really understand what it is for, and their site is doesn't really explain it well either.

What is the advantage of using this instead of something well-proven and battle tested like an in memory SQLite database?
1 comments
victorbjorklund 1008 days ago
DuckDB is a columnar db and focused on analysis queries. So it is more optimized for those workflows. Of course for many things sqlite is fine but if you have a really large db with complex queries duckdb should be faster.
link
filleokus 1008 days ago
DuckDB also compresses the data. I've ingested ≈ 1 TB of apache-like access logs into a duckdb file that was small enough to load completely into ram on my local machine (like 10-15 GB). It was blazingly fast to query over.

(The large spark cluster was still ingesting the data when I was done with my analysis, only working on my local laptop)

link