|
|
|
|
|
by dapomeranz
971 days ago
|
|
In the same way that dbt has established a foundational format for relational data transformation, dlt seems poised to establish a more ubiquitous solution for batch data pipelines. Native upsert "relational-ization" with configurability is a huge challenge that data engineers have been fighting for decades. Obviously, this library will not replace Kafka setups or more intensive streaming jobs, but there are so many custom CRON scripts in the universe doing exactly this. Would be a big win to see some standardization to Python and an OSS tool like this. |
|