|
|
|
|
|
by carlosf
2366 days ago
|
|
Good article! I actually like systemd, but nowadays I generally run stuff as containers, so there is less and less opportunity to use it. Here is my current setup for Airflow: - 1 container for the webserver - 1 container for the scheduler - 1 managed database (I use Postgres, it's a fairly small instance.) - 1 S3 bucket to deploy DAGs. I mount it on my containers using s3fs-fuse. - You can monitor the scheduler using a PID file, whereas the webserver can be monitored probing your admin URL. - Most configuration can be done using environment variables, which is perfect for containers. - I also configure DAG logs to be shipped to a S3 bucket. |
|