|
|
|
|
|
by andrewmcewen
1772 days ago
|
|
An example of what lineage would look like is the following: You have source tables A and B that are joined to create a model C and then C is used in dashboard D. We are able to infer the lineage A->C->D and B->C->D. We extract lineage in a couple of different ways. The main way is by parsing SQL queries in your data warehouse to determine which tables and dashboards are upstream/downstream. The other way we extract lineage information provided directly from dbt and BigQuery who have nice APIs for this information. We are working releasing an API in Q4 that supports pushing information from say an Airflow DAG to Secoda to give us more lineage context. Hopefully this answers your questions. |
|