Databases can usually be split into one of two types; OLTP (row-based) or OLAP (columnar). OLTPs are used mostly for transactional workloads whereas OLAP is mostly used for analytics.
Here goes...
Take 1: sqlite is to Postgres, what duckdb is to Snowflake/BigQuery.
Take 2: In a similar way that sqlite is an in-process/memory OLTP, duckdb is an in-process/memory OLAP.
I should mention some caveats/exceptions/notes about my statements above:
- there are OLAP projects out there that use Postgres as their basis.
- HTAPs are DBs that allow you to define tables as either row-based or columnar.
- duckdb works with sqlite tables, and its SQL is heavily based on postgres SQL
- duckdb 0.9.0 is being released next week :)
- It seems duckdb is poised to become an almost ubiquitous component of the analytical stack. Apache Iceberg, dbt, ...
In addition to the other examples given, it's a quick way to run sql commmands against a CSV, and lets you quickly export to Parquet as well run queries against Parquet files. If you're going to be using Parquet but want to run some quick tests (think setting up an Athena database) it's nice to do it ad hoc rather than having to setup a full ETL process first.
Here goes...
Take 1: sqlite is to Postgres, what duckdb is to Snowflake/BigQuery.
Take 2: In a similar way that sqlite is an in-process/memory OLTP, duckdb is an in-process/memory OLAP.
I should mention some caveats/exceptions/notes about my statements above:
- there are OLAP projects out there that use Postgres as their basis.
- HTAPs are DBs that allow you to define tables as either row-based or columnar.
- duckdb works with sqlite tables, and its SQL is heavily based on postgres SQL
- duckdb 0.9.0 is being released next week :)
- It seems duckdb is poised to become an almost ubiquitous component of the analytical stack. Apache Iceberg, dbt, ...