|
|
|
|
|
by caravel
3394 days ago
|
|
[Bias Alert: author of Airflow] can confirm that Airflow allows you to incorporate all of the seven steps, and more as an open platform. At Airbnb Airflow is far from being limited to data engineering. All the batch scheduling goes through Airflow and many team (data science, analysts, data infra, ML infra, engineering as a whole, ...) uses Airflow in all sorts of ways. Airflow has a solid story in terms of reusable components, from extendable abstractions (operator, hooks, executors, macros, ...) all the way to computation frameworks. |
|