Hacker News new | ask | show | jobs
by theamk 1436 days ago
"existing tools such as Airflow, Prefect, Temporal, etc, but in most cases they require writing code in a programming language such as Python to define DAGs. [...] there is concern that adding another later of Python code will further decrease maintainability. So we developed Dagu, which require no coding, and is easy-to-use and self-contained, making it ideal for smaller projects with fewer people."

If the goal was to eliminate project-specific DAG code in programming languages, have you considered writing an YAML adapter for an existing tool? For example, an "generic" Airflow pipeline which reads YAML files and generates tasks for each clause. This would have much less code than writing scheduler from scratch, and none of it would be project-specific.

1 comments

Thanks for the comment :) That's a good point. We considered using Airflow in the first place, but we decided not to. Because we had only a few number of developers and felt those tools were overkill for our use case. Also, we couldn't afford those tools' additional overheads such as operational costs or learning curves. We just wanted a better Cron that can be easily installed and operated and is capable of visualizing and running DAGs.