|
|
|
|
|
|
by 0cf8612b2e1e
462 days ago
|
|
|
Where are you using/advocating crabwalk? It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of SQL scripts. Presumably the all-local case needs less complexity. Unless the idea is that this will eventually support more connectors/backends and work as a full dbt replacement? |
|
|
* Built-in column level lineage (i.e. dump in 20 .sql files and crabwalk automatically figures out lineage)
* Visualize the lineage
* Clean handling of input / output (e.g. simply specify @config output and you can export results to parquet, csv, etc.)
* Tests are not yet implemented, but crabwalk will have built-in support for tests (e.g. uniqueness, joins, etc.)
we're using it in our product (https://www.definite.app/), but only for lineage right now.