|
|
|
|
|
by kfk
2366 days ago
|
|
With dask we code the workflow in the notebook and run in the notebook. We don’t have to fiddle with operator as every task is python code. Dask is easy to install which is important since each analyst has to be able to test the workflows before sending them to production. Finally by programming our own scheduler we can build the things we need. For instance we are able to listen to sql tables and api changes and trigger work based on that. Anyway I am sure I could make Airflow work too but it’s a harder fit vs dask. |
|