Something I've always wondered... How do downstream DB integrations like Fivetran or other ELT services play with SQLite? It seems like that's another consideration for running it in production at a real company.
I don’t think most touch it. As much as I’ve always thought embedded dbs have a place in data analytics and warehousing, I think the issue is that it’s really a sql api on top of a file system and it’s easiest to just copy the db with something like rclone and treat it like a file. Like others mentioned there’s also litestream for replication.
Fivetran, I believe is still involved and committed to duckdb. Duckdb has a SQLite extension. So, it’s entirely possible they could add SQLite support. I would say the issue then is familiarity and demand within the industry.
Fivetran, I believe is still involved and committed to duckdb. Duckdb has a SQLite extension. So, it’s entirely possible they could add SQLite support. I would say the issue then is familiarity and demand within the industry.
https://www.fivetran.com/blog/how-fast-is-duckdb-really
They also have a talk given at duckcon that goes over the architecture.